DuckDuckGo Will Automatically Encrypt More Sites You Visit

It’s increasingly common for the data that passes between your browser and a website’s server to be encrypted with HTTPS, which makes it impossible for outside snoops to read. But you don’t get that protection if the URL drops that crucial “S” after HTTP. And while some mechanisms do redirect you to an encrypted version of a site, they often do so only after exposing that initial request. The makers of the privacy-focused search engine DuckDuckGo think there’s a better way.

Today DuckDuckGo is releasing a feature called Smarter Encryption that combines its existing private search capabilities and tracker blocking service with a new tool to upgrade encryption for more of the sites you visit. It’s available on DuckDuckGo’s mobile browser for Android and iOS, and through the company’s desktop browser extension for Firefox and Chrome. DuckDuckGo is also open sourcing the code behind the feature so other sites and platforms can adopt it as well. First up? Pinterest.

“I think people tend to think it’s a less of a problem because a lot of sites automatically redirect you to an encrypted version now, though a lot of sites also still don’t,” says Gabriel Weinberg, DuckDuckGo’s founder and CEO. “We wanted to give people a more comprehensive privacy solution no matter where the internet takes you.”

DuckDuckGo isn’t the first organization to tackle the HTTP upgrading problem. The Electronic Frontier Foundation’s HTTPS Everywhere browser extension and Chromium’s HSTS Preload List provide similar functionality. The latter is enabled by default across Chrome, Firefox, Safari, Opera, Edge, and Internet Explorer. These offerings all function basically the same way, working off a list of sites that offer HTTPS versions to upgrade connections before they’re established. But DuckDuckGo’s tool has one major difference: Rather than populating a list of upgradable sites manually, Smarter Encryption fills it out automatically using the same web crawling smarts built into DuckDuckGo’s private search service. No one needs to add and remove entries from the list on their own; whenever the crawler sees that a site supports HTTPS, it records that as the default for all visitors using Smarter Encryption going forward, regardless of what URL they type or link they click.

This automatic element makes the list strikingly comprehensive. Compared to other tools, which have fewer than 150,000 sites on their preload lists, Smarter Encryption already works on 12 million sites, making it more likely that you’ll reach for the encrypted version of a given site from the start.

Weinberg says DuckDuckGo’s auto-populating strategy wasn’t as easy to build as he first expected, because of the patchwork of encryption implementations on the web. For example, some sites are only set up to encrypt some of their pages. This means that if Smarter Encryption tries to upgrade your connection to that domain, some functionality and pages may break. It took a number of workarounds—including developing visual tests to automatically assess whether a page looked different after adding encryption upgrading—to make it all work without any browsing disruptions. DuckDuckGo launched a beta of the tool in 2018 to test for any issues. And now it’s finally ready for prime time.

Search engines and social networks are prime platforms for adding encryption upgrades, because they both incorporate huge numbers of links that are crawler or user-generated and may not include “HTTPS.” Pinterest itself is fully encrypted, but implemented Smarter Encryption to protect its users as they click links posted on the platform that lead to outside sites. Pinterest says that after incorporating DuckDuckGo’s feature, about 80 percent of outbound traffic routes through HTTPS, up from 30 percent before.

“DuckDuckGo was the perfect fit for us because they maintain a comprehensive list of upgradable sites, generated by comparing the HTTP and HTTPS version of a site, and adding a site to the HTTPS upgrade list if the two versions are identical,” Pinterest explains in a blog post shared with WIRED. “We can then regularly pull and ingest their list.”