03/18/21
- Made site analytics/tracking concerns worth slightly higher for privacy score to further punish VPNs who implement invasive trackers within their apps and programs. Partially rebalanced the category to accommodate it.
- Added infrastructure & client audits to security & privacy categories and rebalanced each category to accommodate it. This is another important indicator to evalaute overall trust in a service.
- Added if VPN servers are owned/rented to the privacy score for another indicator of overall trust in a service.
05/09/20
- Moved “helps privacy causes” to the Additionally Address section as this is both A) Relative and B) Doesn’t directly affect a service in any meaningful way that should affect a systematic scoring method. Moving it to additionally address still allows discussion on the matter without unfairly docking points for forced cooperation.
03/16/20
- Speed Team is officially enacted and will be the source of speed scoring. Speed tests are still the same, but scoring criteria have been updated to reflect a worldwide speed test. Eventually, scoring criteria will also be split or fully relative values to the performance of other services. Ex. Being the fastest VPN we’ve tested will immediately be a perfect score. This is to prevent the guaranteed throttling that will happen with testers using insanely fast internet who will not be able to accomplish perfect scores even if a service is the fastest we’ve ever tested.
03/15/20
- Added multihop support to additionally address.
- Added “Helps Privacy Causes” to privacy score and rearranged criteria to still add to 5.
- Added misleading marketing to BOTH Security & Privacy. This is not exclusive to either category and both categories will have the same score.. Ex. If VPN A has strongly misleading marketing, it will lose all points for misleading marketing in BOTH security & privacy. The criteria score will always be the same. Rearranged scoring to add to 5.
01/08/20
- Chinese jurisdiction is now worth less than the 5 eyes under jurisdiction.
- Security & Privacy category have been rescored slightly, with some new additions such as a ‘Previous History’ score for Security.
- Security & Privacy have received some basic formatting updates and criteria text changes.
- Split tunneling has been added as a feature to look for in Additionally Address by popular demand.
- Added ‘Future Ideas’ to the bottom of this document so the community is aware of upcoming ideas and changes that I’m considering.
01/01/20
- Percentages of the categories have been slightly moved around as Security & Privacy BOTH should be more important than speed. New category percentages in order of appearance are: 25%, 25%, 15%, 5%, 10%, 20% (Previous: 20%, 20%, 20%, 5%, 15%, 20%)
- After updating Privacy scores of old services, it was clear scoring needed to be rearranged because services with a questionable history were scoring higher than services with a perfect history, which should not be the case for privacy. This update includes the new rearranged criteria scoring.
- Removed first-party VPN server requirement under Privacy as not a single service relies on 100% self-owned servers and this is an unrealistic expectation making a perfect privacy score impossible.
12/31/19
- Added version numbers to the review protocol in order to better track how old reviews are. This data will also be used for the VPN Chart on the Techlore website so users know what protocol was used for a review.
- Improved formatting and coloring, especially for category criteria scoring.
- Added middle score for Warrant Canary criteria to encourage providers to use a warrant canary even if they don’t have a whistle clean history.
12/31/19
- Speed team is now FINALLY included as part of scoring! As of today, we do not have many members and the speed team is brand new, so it will only account for 10% of the total speed score, with its results shown in reviews to help users see other country results. As we work out issues and we can guarantee better accuracy with results, its influence will increase until it’s 100% of the total speed score.
- Decreased ping speed requirement to be anything under 15 ms for a perfect ping score. (10 ms previously)
- Decreased download/upload speed requirements to 92% minimum for a perfect score. (95% previously)
- Moved Chinese Jurisdiction to 0 points under Privacy as this has been an increasing concern.
- Added warrant canary to scoring for Privacy, worth 0.2 points max and moving the previous 4 criteria worth 0.25 to 0.2 to make room.
03/27/19
- Changed Ookla speed test to https://speedof.me/ after researching more accurate forms of speed testing.
- Added “Open to interpretation by the viewer.” warnings on any category that may vary by user configuration.
- Improved formatting for Stability category.
11/03/18
- Added Android Studio & Google Drive File Download to speed testing.
- Added disclaimer to speed testing, since I am currently unable to average out world-wide results.
- Tidied up some formatting to make it easier to read.
- Added instructions on how scoring works.
07/18/18
- Added specific point deductions for every requirement to increase transparency on how I arrive at final scores for each category.
- Separated “Security” category into “Security” and “Privacy” split 20%/20%. All previous requirements have been rearranged accordingly between the two new categories.
- Added the amount of personal information needed to open an account in “Privacy”.
- Added website analytics/tracking concerns in “Privacy” category.
- Added login security (like 2FA) in “Security” category.
- Have officially begun a search for a world-wide speed team to average out VPN speeds.
- Removed P2P speed testing for inconsistent results.
- Added a “beginner/intermediate/advanced” label to give services to mark services as easy/moderate/difficult to use in the “Additionally Address” section.
- Added color to the document.
- Added the Techlore logo in the header.
- Added autoconnect to usage.
06/15/18
- Removed WebRTC Leaks from “Security” because this is more reliant on browser configuration than the VPN being tested.
- Removed the ISO download from the Ubuntu website test because of radically inconsistent results.
06/13/18
- Added Speed Consistency to the usage category.