Hi, I recently discovered the .wget-hsts file in my home directory which is used to persist state for HSTS: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security It contains lines such as: # HSTS 1.0 Known Hosts database for GNU Wget. # Edit at your own risk. # <hostname> <port> <incl. subdomains> <created> <max-age> github.com 0 1 1450887745 31536000 ftp-master.debian.org 0 0 1472482586 15552000 diffoscope.org 0 0 1449765396 15768000 reproducible-builds.org 0 0 1471542629 15552000 www.dropbox.com 0 1 1458394011 15552000 reproducible.debian.net 0 0 1448074844 15552000 […] I can't help but think of ~/.ssh/known_hosts which moved to hashing the hostname for various security/privacy concerns. Shouldn't wget make the parallel change? Regards,
URL:
<http://savannah.gnu.org/bugs/?51666>
Summary: Please hash the hostname in ~/.wget-hsts files
Project: GNU Wget
Submitted by: nok
Submitted on: Sat 05 Aug 2017 04:14:03 PM CEST
Category: Feature Request
Severity: 3 - Normal
Priority: 5 - Normal
Status: None
Privacy: Public
Assigned to: None
Originator Name:
Originator Email:
Open/Closed: Open
Discussion Lock: Any
Release: trunk
Operating System: GNU/Linux
Reproducibility: Every Time
Fixed Release: None
Planned Release: None
Regression: No
Work Required: None
Patch Included: No
tags 870813 + upstream forwarded 870813 https://savannah.gnu.org/bugs/?51666 thanks Hello Chris, Am Samstag, den 05.08.2017, 08:59 -0400 schrieb Chris Lamb: ... I forwarded your feature request to the upstream bugtracker. Regards Noel
Follow-up Comment #1, bug #51666 (project wget): The files are read/write only to the user (in case you didn't protect your home directory appropriately). If you are still concerned, what about ~/.mozilla/firefox/*.default/SiteSecurityServiceState.txt and likely hundreds or thousands of other files containing potential private information ? Not thinking of your private keys lying around in .ssh/. And even file dates and sizes are potential private data leaks. While we could hash anything, it would be way safer for you to protect your complete home directory. We could think about an option, just in case you want to keep your HSTS cache in a public space, e.g. /var. But we already think of using SQLite for storing and maybe sharing with Firefox.
Hello Chris, Am Samstag, den 05.08.2017, 08:59 -0400 schrieb Chris Lamb: Upstream Tim answered but the answer didn't reach you. You can read it here: https://savannah.gnu.org/bugs/index.php?51666 or https://bugs.debian.org/870813 Regards Noel
As long as it is optional... It would be nice being file compatible with Firefox (at least reading Firefox HSTS db). Maybe the sqlite backend that has been mentioned earlier should then work with the same settings (hashed/not hashed). Absolutely, but in this special case you open up a can of worms. From a security standpoint, the average home directory is a nightmare. Once someone gets access to it (read or write)... Regards, Tim
*Buenas tardes! Te contacté con respecto al fondo $ 5.2 Millones de dólares depositados en un banco aquí por mi cliente atrasado que llevan el mismo apellido contigo. Además, en esta transacción quiero que me responda confidencialmente directamente para más detalles. Gracias, Jeff Edwards Esq*
I think the changes to ~/.ssh/known_hosts are mainly security theater, and they make ~/.ssh/known_hosts significantly harder to use for other purposes (e.g. "is there some foo.biz server that i've forgotten about?"). there's a ton of other places (e.g. bash history, browser history, etc) where similar information leaks into the local filesystem, accessible by the user. hopefully this perspective isn't too much on the side of privacy nihilism. i'm happy to be called out if it is! but my instinct is that we should avoid making these files more complex than they need to be. If we want to protect the ~/.wget-hsts file or other HTTP client state, we should be using filesystem and other operating system permissions to protect it. It's a good catch that this file should never be world-readable in the user's home directory -- that's a serious leak, due to the homedir often being world-readable by default (e.g. for public folder sharing, dropboxes, and the like). The version of wget that i've got (1.19.2-1) currently just creates it with 777 & umask -- that's a problem: 0 dkg@alice:~$ rm -f .wget-hsts && wget -O /dev/null -q https://www.debian.org && ls -la .wget-hsts -rw-r--r-- 1 dkg dkg 169 Dec 7 17:45 .wget-hsts 0 dkg@alice:~$ But the more sensible solution is what Tim mentions -- clean up the permissions or stash the file in an unreadable subdirectory. More generally, the fact of wget keeping state by default is maybe surprising for some people -- wget doesn't keep cookies by default, for example. Perhaps wget should prefer a state directory where all of its state can be kept by default, rather than having to specify all of --save-cookies --load-cookies --hsts-file to manage HTTP client state coherently? Then we could even use ephemeral filesystems, ephemeral filesystem encryption keys, or other fundamental protections to keep the state protected and private.
Follow-up Comment #4, bug #51666 (project wget):
Hello!
I've written a series of patches (attached) that adds this feature. Please let
me know if there's some other way that I should be submitting these instead.
Some notes:
* I have little security experience, so these should be checked over somewhat
carefully.
* The OpenSSL implementation of ssl_hash_salt uses functions only in OpenSSL
0.9.7 and later. I'm not familiar enough with OpenSSL to get it to support
anything prior to that, but the rest of openssl.c seems to imply that some
versions before that are supported. Sorry about this!
* This patch seamlessly and automatically converts old hsts databases to
hashed databases. Unpatched versions of Wget parse those databases without
issue, though they functionally ignore any converted entry.
** If a patched Wget creates a hashed entry for a HSTS-enabled website, then
an unpatched Wget creates a unhashed entry for the same website, then a
patched Wget converting that entry will create a duplicate entry in the store.
I'm not sure how much of a concern this is.
* This switches the HSTS store from a hash table to an array, and each search
for a host creates a hash for each element in that array until either one
matches or the end of the array is reached. This isn't a problem for
reasonably sized stores; in my test on a 50-entry .wget-hsts, an unpatched
wget was 100 ms _slower_ (presumably due to network jitter). However I can
imagine that if you have several thousand entries this could become a
noticeable slowdown.
(file #44842, file #44843, file #44844)
_______________________________________________________
Additional Item Attachment:
File name: 0001-Add-utility-functions-for-hashing-and-salting.patch Size:4 KB
File name: 0002-src-hsts.c-Hash-HSTS-enabled-domains-to-improve-priv.patch
Size:23 KB
File name: 0003-src-hsts.c-Add-test-for-hashed-domains-stored-in-dat.patch
Size:2 KB
Follow-up Comment #5, bug #51666 (project wget): Thanks for addressing the issue. Saving the salt together with the (salted) hash isn't of big help when we talk about a limited set of input strings. You can get complete lists of existing domains and brute force through them in a few seconds. Can even be optimized by starting with the top 1m domains. I just mention this to make clear that this way of obscuring is far from being safe. It is just slightly more effort to reverse the domain names in comparison to unsalted hashes. Anyways, it helps from being fly-by looked at, e.g. on the console. I would like to ask you to not use OpenSSL for hashing. We have/use the SHA256 digest functions from gnulib anyways. So it should be straight forward.
Follow-up Comment #6, bug #51666 (project wget):
Thank you for your feedback.
Regarding the salt: I used one because OpenSSH uses one and that was both what
the original bug report references, and written by somebody likely more
experienced than me Re: security. If you want me to remove the salting or
something I can do so.
This version of the patch (only one this time, because there's little point in
only separating adding the tests) has the following changes from the previous
patchset:
* Uses gnulib hashing functions instead of gnutls/openssl ones (Sorry, I
didn't know gnulib had those!).
* Fixes the interleaved versions duplicate entry thing.
* Adds dependency on gnulib's crypto/gc and crypto/gc-random (not any
crypto/gc-<hash algorithm>) to generate decent quality randomness for the
salt. Seeing as the salt apparently isn't that important, I guess it could be
switched to random_number if this is a problem.
* In case someone does want to switch to crypto/gc-<hash algorithm> in the
future (to easily add another hashing algorithm or something), the prefix was
switched from 1 to 5 to match the value of GC_SHA256 in gc.h.
(file #44848)
_______________________________________________________
Additional Item Attachment:
File name: 0001-Hash-domains-in-HSTS-database-to-improve-privacy.patch Size:25
KB
Hello Attorney, I was referred to you by the Bar Referral Services. Do advise if your firm handles Litigation Cases. Regards.
Hi,
Due to a server error on your e-mail, (870813@bugs.debian.org) incoming messages were delayed
log on to portal to recover your delayed messages.
Read Delayed Messages
<870813@bugs.debian.org