Reviewing a VPN can be tricky. There's a lot to consider, an array of factors and features to weigh up, and everyone has their own view on what's important, and what really isn't.
As a result, definitive verdicts are hard to find. It doesn't matter how often or how carefully anyone measures connection speeds from the UK to Germany on a Windows laptop – this won't tell you very much if you're interested in connecting from Australia to New York on your Android mobile.
What we can do is check out a lot of different aspects of the service, and tell you more about them. You may not be interested in all of these – perhaps you only ever use the same location, for instance, so couldn't care less about how the server list is organized, or whether there's a favorites system – but hopefully we'll cover enough ground to give you a feel for what a provider is like.
In this feature we're going to explain the key issues we consider, and how we go about evaluating and testing them. Although do bear in mind that while full reviews of the top VPN providers will cover everything we're going to discuss here, shorter reviews will simply focus on the major points.
But it's equally worth noting that what we've laid out here is just a fraction of what might potentially happen in a real review. If we're curious about exactly what a VPN app is doing – maybe we're wondering if it might be malicious, for instance – we might run its executable past a security scanner (try Ostorlab), examine the code of a browser extension or use something like dnSpy to decompile and browse a .NET client.
But going into that sort of detail would require an entire book to cover properly, so there's no room for that here. (No, really – in fact, forget we mentioned it).
Collecting features
The process of evaluating a VPN begins at its website, by collecting details on the service and its features.
Network size matters, though not as much as providers sometimes claim (as long as a VPN has the locations you need, it doesn't matter whether there are 20 or 2,000 more). As well as the numbers, we check to see how widely dispersed the locations are, whether they include countries you won't always get with the competition, or if they leave gaps in coverage elsewhere.
It's equally important to understand the services supported by each server. Do they all support P2P, streaming, OpenVPN, and everything else offered by the provider? This isn't always easy to find out, but it's worth making the effort (that list of 300 locations may look much less impressive if you realize you can only use P2P with three of them).
A good VPN should offer custom desktop and mobile apps, as they'll be the easiest way to use the service. Be careful when you're checking this on a website, though. Providers will sometimes give you a long list of platforms, but although some of these will lead to downloads, others might just point you to a tutorial explaining how you can set up the service manually. Follow each link to find out for sure.
Identifying protocols supported by a VPN tells you a lot about the service. If it covers the key standards – OpenVPN, L2TP/ IPSec, IKEv2 – there's a good chance you'll be able to use it on most devices and platforms (routers, game consoles and so on), even if they're not directly supported by provider apps.
We pay close attention to any details of how each protocol is implemented, too, including the specifics of any encryption and authentication methods which might be employed. These aren't always visible upfront, but there's usually some information tucked away in the support site.
Does the VPN have any interesting bonus features? Common examples are ad, tracker and malicious URL blocking; the provider's own DNS system; Onion support; split tunneling, allowing you to choose which applications use the VPN, and which don't; and maybe Bitcoin support for payments, improving your anonymity when you sign up.
Checking out the price helps us to understand whether the service is good value, but there's more to that than looking at a single headline figure. Some VPNs don't offer their lowest price unless you sign up for several years, so it's important to look at the range of plans and prices on offer.
If collecting all this data sounds like it could be a lengthy and tedious process, you're right; it is indeed. But it's not just about finding figures and feature lists. Just the experience of looking for the relevant information will give you a good feel for what a VPN is like.
A dubious provider might be short on detail, for instance, with a poorly organized website making it hard to find what you need. Information might be inconsistent or outdated, and you'll sometimes see a little marketing trickery, such as making unrealistic speed or website unblocking promises which you suspect the company can't deliver.
But a professional provider will have a well-designed website which makes all the most important information visible upfront, with plenty of technical detail tucked away if you want it. Overselling will be kept to a minimum, and honesty and transparency is likely to be the order of the day.
- Check out our list of the best VPN providers in the market
Privacy
Assessing the level of privacy offered by any VPN is difficult, as you're relying on the provider to honestly tell you what they're doing and how the service works. Still, there's often enough information to point you in the right direction.
This starts with the technical data you collected earlier on the protocols supported, the encryption and authentication. OpenVPN support is best, IKEv2 not far behind, L2TP/IPSec is acceptable, but the outdated and insecure PPTP is best avoided entirely.
We might look for AES-256 data encryption, RSA-2048 or 4096 to cover handshaking, and Perfect Forward Secrecy to generate new keys for each session – but the truth is that VPNs don't always fully spell out what they're doing.
It's sometimes necessary to browse the support site for clues, or you can download sample OpenVPN configuration files to give you some idea of how the service works. If the service has live chat, try asking someone; the agents can usually answer pre-sales queries.
App features are key. Quality VPNs will use their own DNS servers and implement DNS leak protection (IPv4 and IPv6) to reduce the chance of your internet activities becoming visible to others. You'll also want to have a kill switch to automatically block internet access if the VPN connection drops.
Beware: just because a service has 'kill switch' on a website feature list doesn't mean that this capability will be available on every platform. Mobile apps often have fewer features and functions than their desktop siblings, for instance, so we check each one to fully understand what it does.
To confirm that a VPN is properly hiding your IP address, connect to any VPN location and point your browser at ipleak.net. If you see your real IP address, or an address owned by your ISP, there's a problem which you need to investigate further. Results can vary according to browser setup, so repeat this in every browser and on every device you use.
We test kill switches on Windows VPN clients by using a custom tool to forcibly close our VPN connection, then we check to see how long the internet remains accessible, and whether our regular external IP is visible to the outside world.
To do something similar manually, use Task Manager to close the process OpenVPN.exe and check whether your internet access goes down immediately, or there's a few seconds delay, and whether it automatically reconnects.
If the kill switch doesn't appear to work, check Settings to make sure it's turned on (sometimes it's disabled by default). Look for alternative settings which might help, too. Some clients include a 'redial if the connection drops' feature which isn't quite as effective, but will still offer some anonymity protection.
Logging
VPN providers, particularly free services, are regularly accused of selling their users browsing history. Most will try to fight back, typically by yelling 'No logging!' on the front page of their website, but years of review experience has told us this isn't always true. That's why we always look a little deeper.
The privacy policy can tell you a lot about a provider, even before you read it. 5,000 words of poorly-formatted, jargon-packed legalese is a bad sign, and so is a 100-word document which tells you almost nothing. What we're looking for is a clearly formatted, well-organized and readable document which allows anyone to find the key details they need.
Those details should be explicit and cover all logging possibilities. A single line saying 'we never under any circumstances log what you're doing online', for instance, doesn't rule out the possibility that the service might record session data: the date and time you log in, your incoming IP address, the VPN IP address you're assigned, the date and time you disconnect and the bandwidth you use. That level of detail could be enough to allow others to link an internet action to your account.
A better policy will rule out this possibility by explaining that it doesn't log connect or disconnect times, incoming and outgoing IP addresses, and so on.
The best policies will also tell you what they do record, why they do so, and what happens to that data. For example, a provider might say that it logs the last connection time and the bandwidth used in that session, but doesn't record incoming or outgoing IP addresses. It could explain that this data is useful to help the company monitor service use and identify inactive accounts (makes sense), but doesn't allow anyone to find out what you're doing online (quite true). And it might tell you that if you no longer use the service, you can email support and they'll delete your old account data entirely.
It's important to keep this in perspective. However well-presented a privacy policy might be, there's no guarantee that anything said in the document is actually true. It's all based on trust.
Checking out the small print can sometimes highlight logging that a VPN is admitting takes place, though. And if nothing else, it can give you clues about how honest and open a provider might be.
We've previously found dubious free VPNs who have copied and pasted someone else's privacy policy, changed any company names in the text and pretended it's theirs, for instance. That's lazy, incompetent, and fraudulent, really – more than enough to tell you that this isn't a provider you should use.
Test server selection
To properly understand how a VPN performs, we use automated tools to connect to multiple servers and test key aspects of the service: server availability, location, connection time, latency and download speeds.
Our tools connect to each server via OpenVPN, and so the testing process starts by downloading the VPN provider's configuration files. (If a service asks us to generate them ourselves, we specify UDP-based connections).
These files are separated into four location-based groups (the UK, Europe, North America and the rest of the world), which we then break down further to choose the servers which will be included in our tests.
Some VPN tests use largely fixed locations: one in London, one in Amsterdam, another in New York, and so on. This is good for maintaining consistency between VPN providers, but it won't necessarily make for a fair comparison.
For example, if SmallVPN.com has a single New York location, and BigVPN.com has 10, then testing one server from each provider keeps life simple. But there's no way to know whether BigVPN.com's single server will accurately represent the good or the bad points of the service.
To try and get a more realistic idea of how a VPN performs, we test up to 30 locations within a group, even if this means they're all within a relatively small area (10 servers in London, say). In real world testing you might be allocated one of these servers, occasionally, so we need to try them all ourselves, if only to see how (or if) they vary.
If there are significantly more than 30 locations, we choose a subset which best represents the group's geographical area. The North America group will include east coast, mid-American, west coast and Canadian locations where possible, for instance.
Of course, this means we may also miss some of the best (or the slowest) servers, but we think it's enough to give us a representative view of the performance in that group.
Performance
Our performance tests begin by using OpenVPN to connect to each VPN provider's test server, in turn. The time taken to make a successful connection is logged, but failed connections are also recorded, and our system tries each server up to three times. It's all very simple, but already we're able to begin to identify connection and server issues.
A follow-up simple ping test gives us an immediate check of latency. Individual figures can't tell us very much, but consistently high or low ping times may be an indicator of service quality.
Our test system next queries a geolocation library, returning and logging the city and country associated with our current IP address. If a VPN provider advertises a server as being in one location, when it's really half way around the world, this may allow us to spot the problem.
Geolocation isn't an exact science, unfortunately, and it's possible that our test results will be incorrect. If we think a server may have a location issue, we manually log on to it later, then verify its country by visiting IPlocation.net. This returns the city, country and other IP-related details as reported by four different geolocation data providers, and if each of these tells you that your VPN server isn't in the promised country, there's likely to be a problem.
Assessing download speeds is even more difficult, but we use several performance testing websites to help. Speedtest.net and Netflix' Fast.com tend to give the most accurate results, but we may also use Openspeedtest.com and TestMy.net to get a second (or third, and fourth) opinion.
We run our tests on a Windows 10 laptop, using a special testing user account to minimize any local bandwidth use (no non-essential background processes running, Windows tweaked to reduce internet access, and so on). The laptop connects to the internet via a wired connection – not a wireless one – again reducing connection variables.
The test sequence then connects to each server in our location groups, logging the results from the speed test sites in each case. The sequence is repeated later, to look for any variations, and we run a single speed check before and after the full test, to confirm that our internet connection is running at its normal speed.
Even with these precautions, there's a strict limit to what these results can tell us. They're giving us a snapshot idea of how the VPN performs from our test location in the UK, but if you're somewhere else in the world, thinking about trying the service some six months later, you may well see different results.
There's a further issue in that our tests use OpenVPN (via UDP) connections. If you're using another protocol – and on mobile devices, you almost certainly will be – then the results may vary.
Speed tests do have a role as part of building up a broader picture of a VPN, though, and our experience suggests they do help to highlight the best and the worst providers.
Unblocking
Many VPNs boast that they can help you get around geographically-based blocking, enabling access to content which you might not otherwise be able to see. Sounds good, but is it true? To assess this, we start by looking at the claims the provider makes.
A good provider will state clearly which services it claims to make accessible (Netflix, Hulu, and so forth), and maybe which it can't. The support site might include articles which explain what you can do if a site remains blocked, and invite you to contact the support team directly if you need help. If you can't access Netflix via ExpressVPN, for instance, you're able to open a live chat session with the support team, ask them which server to use, and get a recommendation within a couple of minutes.
Lesser VPNs might not have any troubleshooting articles, and they'll be less likely to commit to helping you access any particular website. Instead of saying 'if you're blocked, contact us for help', they'll give more of an impression that 'if you're blocked, tough, we can't promise anything, that's just the way it is'.
The most dubious VPNs will look like they're making big promises, perhaps claiming they 'allow you to access ANY website', or displaying Netflix and other logos to imply that they'll be available, without specifically saying that. But dig deeper and you'll either find no background information at all, or just a disclaimer saying they can't guarantee anything.
To test this ourselves, we attempt to access and stream US Netflix and YouTube content from multiple US VPN servers, and BBC iPlayer from all available UK servers. The results are logged, and if they're entirely negative, and the provider suggests they shouldn't be (the company claims we can access Netflix, and our checks say otherwise), and recommends asking for help, we get in touch with the support team to see what they can offer.
Even at this late stage, how an organization responds can be crucial. If the provider can recommend another working server, tell us that it's aware there's an issue and that it is taking action, the firm will earn bonus points. If not, and if the company doesn't live up to promises made on the website, then it will be penalized.
Apps
A VPN can have a huge network of fast and reliable servers, but that's not much help if its apps make the service difficult to access or use. That's why we place a lot of importance on a provider's software, its features and usability.
We expect a VPN to provide its own apps for Windows, Mac, iOS and Android. Support for other platforms isn't as important, but does give us valuable information about the provider. The vast majority of users won't care if their VPN has a Linux client, for instance, but its existence alone tells us the provider has resources, real VPN experience, and is doing its best to reach and help the maximum possible audience.
Some services also offer browser extensions. These are generally more limited than apps, as most only protect browser traffic, and not other programs running on your device. But there are some exceptions, so for instance, ExpressVPN's Chrome extension can communicate and control the native ExpressVPN client on a PC.
App usability is key. It's also very subjective, but we try to take the most important factors into consideration.
The opening interface should make it easy to find the server you need, or reconnect to commonly used locations. A simple alphabetically sorted list, with a search box and a favorites system, could do the trick. Apps with clumsy map systems, or which force you to make multiple clicks to get connected, will usually be marked down.
Apps will usually offer a more sophisticated location picker, and we'll score that on the features offered, and their usability. A good app might allow you to filter locations by continent, for instance. We like to see some indication of server speed (ping times, server load figures), and if you can sort by those to highlight the fastest, that's even better.
The ability to easily find and reconnect to commonly used locations is very handy, especially on mobile devices where it's harder to spot items in a very long list. A common trick enables marking locations as favorites, usually placing them on a separate list at the top of the screen. ExpressVPN goes one better by also offering a 'recent' tab, where you can find and connect to recently used servers in a couple of clicks.
It's vital that a user knows when the VPN is protecting them, so the app should display notifications to show when it's connecting, connected, or when the connection drops. Ideally the interface should make this connection state very easy to spot, for example with the entire app window changing color to indicate when it's connected. Not everyone wants this level of interaction, though, so the very best apps will also give you the option to customize the notifications you see.
We also check how easy it is to switch servers while you're connected. Some apps make this surprisingly difficult, perhaps refusing to even display the server list until you manually disconnect. We prefer apps which make the process much more natural, where you can view the location picker as normal, select some other server, and the app closes the current connection and establishes a new one.
Many apps include an 'automatic' or 'fastest' selection mode, where in theory they choose the best server for you. This doesn't always work as you expect, though, so we usually check it to confirm the results make sense.
We measure performance in detail using our own test environment, but this can also vary by app. That's why we also check connection times, browse websites, and generally evaluate the real world experience on each device.
Settings matter, too. At a minimum, we want to see a kill switch, ensuring internet access will be blocked if the connection drops. A 'redial' option is less effective, but has similar results. The ability to change protocols shows us some versatility. An app which can automatically connect when you access new networks is valuable, as you won't have to remember to do it yourself. And a 'connect on device start' option is almost as useful, especially if you can choose the default location (a specific server, the last one you used, whatever it might be).
Additional geeky touches, such as the ability to add your own custom OpenVPN settings, are welcome, and can be hugely powerful. But we don't place too much weight on these in scoring terms, as the vast majority of users will never pay them any attention at all.
Finally, we look for settings which allow us to fundamentally change how the app works, and check to make sure that these all have sensible defaults. Many users won't spend much time exploring their options, so it's important that each app works as you'd expect from the moment it's installed.
Support
VPNs can run into all kinds of hard-to-diagnose issues, and it's important that every provider offers fast, accurate and reliable support.
The website should include plenty of helpful setup, usage and troubleshooting guides. The installation section needs tutorials for multiple platforms, for instance, and should cover as many operating system versions as possible. A generic Windows setup guide is okay, for instance, but having specific articles for Windows 7, 8 and 10 is better.
Troubleshooting advice should be thorough, and demonstrate genuine expertise. We might search any web knowledgebase using common keywords, such as 'speed', then browse the results. Poor providers might not give any useful hits. A better service will return articles on topics like improving performance, though with only basic information which contains nothing you couldn't figure out already ('connect to a closer server', 'try again later').
But the best providers will expand every idea to give you more than you expected ('here's how to use the client's displayed ping times to find a closer server'), and leave even the most experienced users with a better understanding of the problem than they had before.
A web knowledgebase can never tell you everything you need to know, so it's important that you're able to get real, live, human support. Ideally this should be available 24/7, with live chat as an option, but don't rule out email or ticket support. This has its advantages – you're able to think more about what you say, before you say it, and attach logs or screenshots to help explain the situation – and the best providers sometimes respond to questions within a few minutes.
We try out the live chat or email support of each VPN provider with a sample question, just to get a feel for how it performs. As with many other aspects of a review, this kind of one-off snapshot can't tell us everything we might want to know about a provider's hand-holding abilities.
We've found it often gives us interesting information about the service, though – an ultra-fast response here, a very unhelpful one there – and it all helps to build a more detailed and accurate picture of how good (or otherwise) a VPN might be.
- We’ve rounded up the fastest VPNs for outright speed
from TechRadar - All the latest technology news https://ift.tt/2KuDv3h
No comments:
Post a Comment