Bye Spy

Method

How Bye Spy tests your browser

A browser privacy test that plays the tracker: what it measures, how the two halves of a run work together, and what a result can and cannot tell you.

Most privacy settings are promises: a switch that says it blocks trackers, a mode that says it hides you. Bye Spy checks the promise by doing what trackers do, from both sides of the relationship, and reporting what worked. These pages document every check: what it measures and how, why the technique matters, how it is used on real websites, how common it is, and how its result is scored.

Threat model

The adversary Bye Spy models is the one that dominates the web: a third party embedded on many unrelated sites, an advertising network, an analytics service, a social widget, which wants to recognise the same browser on each of them and build a profile of where it goes. A second, weaker adversary is the first party with long memory: a site's own analytics, recognising you on every return visit for years.

To link visits, such a tracker needs an identifier that is the same in two places. Mayer and Mitchell's survey of tracking technology divides the ways to get one into two families[1], and Bye Spy's two heaviest areas follow them:

  • Stateful tracking stores an identifier in the browser (cookies, storage, the cache) and reads it back. See Cross-site tracking.
  • Stateless tracking derives an identifier from the browser's properties without storing anything. See Fingerprinting.

Around these sit blocking, which keeps the tracker from running at all, and smaller channels that carry an identifier or context beside them. Out of scope are adversaries a web page cannot observe: your network operator, malware on your device, data a site passes to partners server to server, and anyone who knows you by the account you sign in with. What a web page cannot test covers these.

Terms

Site
A scheme and a registrable domain: https://heysec.com and https://byespy.heysec.com are the same site, https://byespy.heysec.eu is another. The registrable domain is computed with the Public Suffix List, so alice.github.io and bob.github.io are different sites[2][3]. Browsers draw their privacy boundaries around sites, not origins.
First and third party
The first party is the site in the address bar; anything it embeds from another site is a third party.
Partitioning
Keying a third party's state by the top-level site it was embedded in, so that the same tracker gets a separate cookie jar, storage and cache on every site[4].
Linkability
Whether two observations, on two sites or at two times, can be attributed to the same browser. It is what tracking needs, and what every check ultimately asks about.

How a run works

Bye Spy runs on two different registrable domains: the site people visit, byespy.heysec.com, and a test tracker, byespy.heysec.eu, which behaves like a real third-party tracker but only ever records random values made up for one run. A run goes through six phases, shown on screen as it happens:

  1. What every site sees. The site's server reports the address and headers it received; the page asks for detailed device hints the way any site can.
  2. Fingerprint. The page computes eleven fingerprint components and reads battery, network and keyboard APIs.
  3. Plant. The test tracker, embedded in an invisible frame, sets third-party cookies and writes a random token to every storage mechanism and the HTTP cache.
  4. Visit the tracker's own site. A small window opens on a page of the test site, stores a value in window.name, and navigates to the tracker's own site as a top-level page, carrying 20 click IDs in its address. There, the tracker reads back whatever it can, takes the same fingerprint, reports to the server and closes the window.
  5. Leaks. WebRTC candidates, referrers, a link ping, timer precision, Topics and cookie lifetime.
  6. Blocking. Ad-shaped bait files from the test tracker and, if you ticked the box, 30 real tracker servers.

The page and the tracker's window never talk to each other directly. They compare notes on the server, under a random 128-bit run ID, and what they store there is deleted when the run ends and expires after 15 minutes regardless. Each check is evaluated as soon as the phase it depends on is complete, and its result is added to the report's file on screen.

Why a window, and not a second frame

Partitioning keys state by the top-level site. A second frame on the test page would share its partition, and a tracker in it would read back everything, proving nothing. To read state as a first party, the tracker has to be the top-level page, just as it is when you visit the social network's own site after reading an article with its button on it. That is what the window recreates. It is opened from your click, so pop-up blockers allow it, and with noopener, so that nothing ties it to the test page. If a blocker stops it anyway, the page offers to open it again or to skip the cross-site checks.

Checking the two domains really are two sites

Every cross-site result depends on the browser treating the tracker as another site. The tracker's frame reads the browser's own verdict from the Sec-Fetch-Site request header, one of the Fetch Metadata headers browsers attach to requests[5], before any cross-site check is scored. A browser that doesn't send the header gets a note that cross-site results assume the domains are different sites.

Measuring the browser, not the page

A test page can easily measure itself instead of the browser. Bye Spy avoids that in a few deliberate ways:

  • It sends what a real site would, and nothing that would change the answer. The test page sends no Referrer-Policy, so the referrer check sees the browser's default; the pages the tracker window passes through send no Cross-Origin-Opener-Policy, which would clear window.name by itself.
  • Both halves run the same code. The fingerprint is computed by one script on the test page and on the tracker's site, and compared component by component, so any difference comes from the browser.
  • Every check has a control. Two ordinary files load beside the ad-shaped ones; utm_campaign rides beside the click IDs; the cache file is fetched twice before the cross-site attempt; two impossible font names test whether font measurements are randomised; a solid colour tests whether canvas pixels carry noise. When a control fails, the check reports Failed or Can't measure instead of crediting a protection that didn't happen.
  • A failure is a result. Browsers often block an API by making it throw. Every measurement catches its own errors and passes them on, and the report shows what the browser said.

Reading a result

A run measures one browser profile, with its extensions and settings, on one network, at one moment. Run it again after changing a setting, in a private window, or in another browser, and compare. Three things are worth keeping in mind:

  • Extensions change results in both directions: a content blocker improves the blocking checks, and an extension that blocks the test tracker's frame makes the cross-site checks fail rather than pass.
  • Some protections are probabilistic or delayed (randomised fingerprints, state deleted after days without interaction) and look different from run to run.
  • A Safe result means the technique as tested did not work. Real trackers combine techniques and adapt; the report is a measure of exposure, not a guarantee.

How the score is made explains how the results add up to a number.

The test's own privacy and security

  • Nothing that identifies you is kept. Every identifier the test plants is random and made for one run. Test cookies live ten minutes and are cleared at the end of the run, as are the tracker's storage and what the run stored on the server. The report on screen exists only in your browser; the PDF and Markdown reports are made there too.
  • Results are kept as anonymous statistics. When a run ends, the page sends what each check found (its status and score) and the total score, for statistics about browsers in general. The server adds the names of the browser and operating system, the browser's major version, and the month. That is all a stored result holds: no IP address, user agent, run ID or fingerprint, none of the values the checks saw, and no time of day. Results are filed under a random key, not in the order they arrive, so a stored result cannot be matched to the request that sent it or to a visit.
  • No third parties unless you ask. The page loads no outside fonts, scripts or images. The real tracker servers and the public STUN server are contacted only when you tick the box, without cookies and without a referrer. The page's Content-Security-Policy lists the same servers, so the page cannot reach any other host.
  • The tracker works for the test only. The endpoints that set or read its cookies answer no origin but the test site, its frame can only be embedded by the test site, and the report its window sends back is refused outright if it is larger, deeper or differently shaped than expected, never trimmed to fit.
  • Visits are counted separately. Visits to the test and these pages are counted with heySec's own cookieless analytics, which never receive a result. They get the path without its query string (so never a run ID) and skip browsers that send Global Privacy Control or Do Not Track.

Bye Spy builds on earlier public tests. The EFF's Panopticlick (2010), relaunched as Cover Your Tracks in 2020, measures how unique a browser's fingerprint is among its visitors and whether it blocks tracking requests[6]. AmIUnique, run by researchers at Inria, collects fingerprints for research and compares yours with its dataset[7]. PrivacyTests.org runs an open-source suite against the default configuration of many browsers automatically and publishes the results side by side[8].

Bye Spy differs in emphasis. It tests your browser as you actually use it, extensions included, rather than a default build. It plays both the publisher and the tracker, so that its cross-site checks observe an identifier actually crossing between two sites rather than inferring it from a feature list. And it judges a fingerprint by whether it links two sites, not by how rare it is in a dataset of volunteers.

References

  1. J. R. Mayer and J. C. Mitchell. "Third-Party Web Tracking: Policy and Technology." IEEE Symposium on Security and Privacy, 2012. ieee-security.org/TC/SP2012/papers/4681a413.pdf
  2. WHATWG. HTML Living Standard, "Sites." html.spec.whatwg.org/multipage/browsers.html#sites
  3. Public Suffix List. publicsuffix.org
  4. Privacy Community Group. "Client-Side Storage Partitioning" (explainer). privacycg.github.io/storage-partitioning
  5. W3C. "Fetch Metadata Request Headers." w3.org/TR/fetch-metadata
  6. B. Budington. "Introducing Cover Your Tracks!" EFF, 19 November 2020. eff.org/deeplinks/2020/11/introducing-cover-your-tracks
  7. AmIUnique. amiunique.org/about
  8. PrivacyTests.org. privacytests.org/about

See what your browser gives away.

Run the test