I came upon an odd question on twitter, asking if it’s considered dark-pattern to use images in order to track user’s activity to which I replied that “#GDPR was meant to prevent website collect data about us; instead it added the annoying “to use this site you must give us your data”, while the alternative path would have been a voluntary “This site does not collect data on its users” with a nice badge that gets better seo-rank“. This sparked an interesting conversation with two developer-advocates from a big search-engine company who questioned the technicality of “what is the cut-off point after which it is considered a dark pattern to collect user data”.
While being aware that their company’s primary source of income is the data collected about its users I was still hoping that these two incredibly smart gentlemen would agree with me, at least on the principle about the value of one’s personal information (aka privacy). I don’t want to regurgitate the importance of privacy here as this topic is already churned, but I would like to address the technical-question from my limited understanding of how search-engines actually work, at least on a theoretical level.
I’m going to make an educated assumption on how search-engines index websites that some javascript code is run and processed. If initially crawlers would scrape the html file, find all visible links and head off to scrape them as well, now the process includes actually loading the page with its onload-triggered javascript. I’m assuming the crawler won’t try to use any input field (although it can see they exist) and won’t try to scroll or resize the window whatever content appears after these javascript-events are triggered is being hidden from the crawler. It’s worth mentioning webrawlers check both mobile and desktop versions in case they have different content. it can try to find js-powered link tags and try to click on them to see whether they lead somewhere but for all purposes it’s more than what I need right now.
Let’s imagine that the search engine results will show a small ranking the site’s respect for its users’ privacy, ranking from 0- full privacy to 6- information is shared with 3rd parties:
Level 0 : Completely static information
With no ajax requests nor data sent to the server this would be the safest kind of web-page.
Of course it means that it can’t even have an internal search engine so it has its caveats, but it would be very easy for the web-crawler to scan all the initial code to see whether it wishes to load additional content via ajax and in such case the page will lose its prestiged ranking.Of course it goes without saying that merely by using 3rd-party-analytics will automatically disqualify the site.
Level 1: Information serves dynamically based on user-interaction
Still nothing about the user is stored, but AJAX and form-fields are allowed, however the js-code doesn’t include any reference to cookies or localStorage. At this level, we can allow having a search engine on the page. As the crawler can’t really identify what is being sent or even when, we need to trust the site, but peculiar behaviour can still be identified as “dark-patterns”.
Level 2: Data is saved locally
Local cookies (as opposed to server-side cookies) allows us to save preferences or even some sort of activity-history but there’s no real need to share it with the server. We started to lose our anonymity as the website remembers information linked to the machine we’re using, but at least it still doesn’t identify who we are.
Level 3: Non-identifiable information is saved remotely
This level is a bit tricky as we, or the crawler, cannot really tell what is saved (and we already said we can’t even observe what is being sent to the server). But I would aspect “honest” analytics to operate at this level; session-cookies are allowed and the site owner should be able to track the anonymous user’s navigation through the website.
Level 4: Transaction-based information is stored
Whenever you make an online purchase, even as a guest-user, you’re forced to leave identifiable information related to the transaction. This means that any e-commerce website cannot be at a better level than this level. But it’s important to notice that beside the specific transaction, the site-owner shouldn’t be able to know anything else about the user’s behaviour (for example, other products he examined before deciding on a particular product).
Level 5: identifiable information is stored
If the site can identify a particular human-being (as opposed to a specific machine), a login is required. In this scenario the site may store whatever information it wants about the users who signed-up and by doing that gave their consent.
Level 6: information is shared with 3rd parties
I initially thought that his level is declarative only (and crawler can’t detect it automatically) but in reality it’s quite easy to find famous analytics – aggregation tools (such as google-analytics) that explicitly mean that, and if sites cheat on their ranking they can always be severely penalise to make them think whether it was a good idea
Site owners who’re currently using his analytics to provide better service for his users might reject my idea as it will impede their capabilities but it’s important to remember that this power comes at the price of the users’ privacy and the fact that they users aren’t aware they’re victims doesn’t make it ok. Also it’s important to point out we’re limiting the tracking of the users and not of the site-activity: it’s still ok to log which page is visited the most or what search-queries were performed. However, It’s not ok to query which product sells the most to American 12 year-old girls.
Yes, it’s possible to cheat the ranking system, and even quite easily; (by css-hacking for example), but I think it’s important to remember how sites used to add invisible keywords to cheat search-engines were penalised for it. If privacy-ranking proves to be important to the users (as it should) google can penalise dark-patterns when detected and detection techniques will evolve over time.
So to answer the original question – yes, it’s technically possible to analyse automatically how well privacy is respected by different websites. Two solutions that already exist and worth mentioning are PrivacyScore that analyse services terms and conditions by using a chrome extension and PrivacyMonitor to analyse the website communication in a bit cruder level. Both solutions require the users to be proactive, but it’s definitely a step forward and to show that it is feasible.
And lastly, comes the question of the self-interest and responsibility of a company whose core business model is selling their users’ personal information to foreign governments (and by allowing other businesses do the same), but I think it’s the responsibility of the good-hearted engineers who works in such companies to warn their users of the danger they face whenever they use a site that harvest their personal information.