It happens to all of us.
You’re asked to look at a website and you don’t have access to your anonymizer. Maybe the undercover laptop is at work. Maybe your proxy is down or running slow as molasses.
So, what do you do?
In today’s post, we’ll explore the role of HTTP Headers and how they keep us anonymous… or not.
When in a pinch, some investigators use browser add-ons to alter their location. These add-ons don’t connect them to a proxy. They send the information that suggests a proxy exists.
The question is: Does it work and is it safe?
XFF is an HTTP header field that websites and browsers exchange to identify each other’s location. Headers work behind the scenes while you surf and you can view them live using the tools at the end of this post.
Browsers use headers to tell the server about themselves. It can say, “Hey, I’m a Firefox browser and English is my preferred language. Here’s my IP address.” Website servers reply with their own headers. Each element on a web page has a header from images and scripts to videos and other files. So, in our example, a video header might tell the browser, “Hey, I was uploaded or last modified on this date, my file is this big, and I’m coming to you from this IP address.”
The XFF is one of the IP addresses that each can share. The XFF is a message of its own and it says, “Hey, I’m not the original IP address. I’m actually coming to you from this location.” When the XFF comes from a website it suggests you’re not seeing the actual hosting location. Our video probably passed through a caching server like a CDN or a firewall. Likewise, when the browser passes an XFF to the server, it’s saying, “I’m not making a direct connection.” This traffic is probably passing through an intermediary, as well, like a proxy connection.
IP addresses and locations are important to servers. Servers use it to deliver specific content to a user, to block unwanted users, and to block a business network or whole country.
The example below illustrates what an XFF looks like when it is received from a website. We’d find the same results if we had tried to look up the website through Domaintools or Centralops. The hosting location is hidden behind a CDN server that delivered the page to us. That CDN is on the CloudFlare network.
The browser add-ons we’re testing mimic a connection known as a transparent proxy. Think of this as the honest proxy. This proxy is transparent in its operation and it adds the XFF to each request you make.
The transparent proxy also announces your original Internet connection by adding an X-REAL-IP or REMOTE-ADDR for remote address to the header. The website’s server recognizes the XFF and the real IP address and it logs the latter in the visitor logs. To see what this looks like, install one of the Firefox add-ons to your browser and run a test on Whoer.net. Your results should highlight the XFF like our test below.
Fortunately, many of the VPN services you’ll encounter are not transparent. They’re known as elite or anonymous proxy servers. Unlike the transparent proxies, anonymous proxy servers choose to withhold the XFF header. Yes, location information may be leaked in other ways, but website servers largely rely on your header and the information proxy servers add to it.
The Firefox add-ons we tested clearly increase the likelihood of standing out. It suggests you’re on a proxy and the server captures your real address. So, why would anyone use it?
Most webmasters will tell you that they don’t look at the server logs. It’s a lot of information, it’s not easy to access, and it lacks the metrics they want to review: visitor behavior. This is why they install analytics. These services tell them how the visitors got to the website, where they went on the website, and in some cases what they searched for and bought. You pass through many of these services every day and they include Google Analytics, Statcounter, Piwik, and Mint.
There is always an exception and Piwik is one of them. Piwik analytics are hosted by the website operator who can also customize what data Piwik collects. Piwik is focused on providing accurate locations, so one of these customizations enables webmasters to discard XFF and instead log the X-REAL IP address. Piwik can also import server logs and, as mentioned, the real IP address is already there.
That’s the downside to analytics. Another downside to using XFF add-ons is consistency. Vigilant operators are probably more likely to note discrepancies in the IP addresses of their users. Tweaking an XFF could also get you blocked on a payment network. Fraud detection services look for proxies and, if found, your payment card, your original IP address, and the related billing information can all get blocked by every merchant employing that detection service.
There’s a lot to consider about the way you share IP address information.
Find out what your browser has to say today.
- Websites and browsers exchange location info via HTTP Headers
- Test these Headers before using a new tool like a proxy or add-on
- Knowing how your tools work helps you know where to use them