#765068 w3m: Misleading Option String for Cookies

Package:
w3m
Source:
w3m
Description:
WWW browsable pager with excellent tables/frames support
Submitter:
Markus Hiereth
Date:
2023-08-09 13:24:03 UTC
Severity:
normal
#765068#5
Date:
2014-10-13 09:22:44 UTC
From:
To:
Dear Tatsuya,

as German translator and after some correspondance, I'm of the opinion
that source file rc.c (still true for Version 0.5.3-17, Line 207)
gives misleading information:

#define CMT_COOKIE_AVOID_WONG_NUMBER_OF_DOTS N_("Domains to avoid [wrong number of dots]")

The msgid would be understood in a way round that the user shall type
in a list of domains to be avoided, wheras README.cookies explains

  If the number of "." in domain name is lesser than 2, it is assumed
  as invalid cookie (cf. RFC 2109 4.3.2), however, you can use
  cookie_avoid_wrong_number_of_dots to avoid this restriction. You can
  set this in "Domains to avoid [wrong number of dots]" on the Option
  Setting Panel.

According to this paragraph, this options item makes w3m accept
cookies that would be rejected otherwise.

Therefore, please consider the following msgid

#define CMT_COOKIE_AVOID_WONG_NUMBER_OF_DOTS N_("Do not reject cookies having the domain attributes")


The README.cookies would need beeing updated the same way

  If the number of dots in domain name is lesser than 2, it is assumed
  as invalid cookie (cf. RFC 2109 4.3.2). However, you can use a
  configuration parameter "cookie_avoid_wrong_number_of_dots" In the
  option panel, besides "Do not reject cookies having the domain
  attributes", the parameter takes a list of strings for the domain
  attribute. Cookies matching these domains are accepted though they
  fail the check described above.

Yours sincerely
Markus

#765068#10
Date:
2014-10-15 09:21:43 UTC
From:
To:
We believe that the bug you reported is fixed in the latest version of
iapws, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 765068@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Alastair McKinstry <mckinstry@debian.org> (supplier of updated iapws package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Mon, 13 Oct 2014 20:10:33 +0100
Source: iapws
Binary: python-iapws python3-iapws
Architecture: source all
Version: 1.0.5-2
Distribution: sid
Urgency: medium
Maintainer: Alastair McKinstry <mckinstry@debian.org>
Changed-By: Alastair McKinstry <mckinstry@debian.org>
Description:
 python-iapws - Python implementation of the international APWS-IF97 steam tables
 python3-iapws - Python3 implementation of the international APWS-IF97 steam table
Closes: 764977 765068
Changes:
 iapws (1.0.5-2) unstable; urgency=medium
 .
   * Include setuptools, python-scipy. Closes: #764977, #765068.
Checksums-Sha1:
 83893b6444223c14304a9aa4471da551f69d0b1f 1831 iapws_1.0.5-2.dsc
 ce780fc5b73cb81a2fbf6860ae2f73c02b76e029 2908 iapws_1.0.5-2.debian.tar.xz
 2dcfecd6900eb0188ef6e0cd6d5d12e97b335772 51082 python-iapws_1.0.5-2_all.deb
 633ebb14115eb9c3bf76f4034b41326d31569f83 51032 python3-iapws_1.0.5-2_all.deb
Checksums-Sha256:
 15cae41d57999a9cdce7b0ab5cd38147bf3726a1b3bef80b5cfe28e7f0da38b1 1831 iapws_1.0.5-2.dsc
 830c8da9e3af61a545e15d1da24b2d108eede89799b1e84dc1f2c274b8c9bb6d 2908 iapws_1.0.5-2.debian.tar.xz
 af85e5a715941aded25b9537b5c7a8852b577d5fedcb43582321638c7021d2a2 51082 python-iapws_1.0.5-2_all.deb
 f8032aa797b00141c7a3bc598a06953ad927829307c62165b220a774dd2c7e6b 51032 python3-iapws_1.0.5-2_all.deb
Files:
 2ce2234f0f7198f28fe9c718ce192588 1831 python extra iapws_1.0.5-2.dsc
 a2635e6685e921260cac9b08fe550047 2908 python extra iapws_1.0.5-2.debian.tar.xz
 11b690cb93bf83f9d7a9d754c728474d 51082 python extra python-iapws_1.0.5-2_all.deb
 094b64962f55fa578f7f03f5edbfd8da 51032 python extra python3-iapws_1.0.5-2_all.deb
iQIcBAEBCAAGBQJUPhIcAAoJEN9LdrZRJ3QsogsQALd2kfWNybFqCVDV4jNM7sdc
VMxpT065oY01ZmjfEukw5x+E9C/psQshFlXmbCEiJR4f5x+q3tur8BhsohK0IycP
RU0/5UOw81pLLNzLQLlb1TaHeZmtNCrhs4oX1r1tguHtiO3/veXMCTVm+16OMBVP
rXS75qP3v4lgFrgTOfuehkAtlEyUJI/GEuE/EQUkpCILaHRC8LG+2PDKuVYzvwmY
HMOUqWeL2Q9kNPICb7WXjrZnCy1mzXswRz/leH9CbfAJvTzLH+ensNtFRxYeLie9
QZvyMgHW/Pd/Uj35VLiBUm5v56IiHgX1kbsZ2T79jURuNWgI7nE/cpbOnXD0H5RN
PePaQaJXihu8VkQ73GpkIM7MFky2G3szjKtkbak8Va2oZz0UYo+LFo/34RrT7c8b
zKWzEke+2ywMsCQ5t/ddhhGxkaYZrYGMmb0Tgp7Ibp5nufEua/ApGLwjlMboInNG
yfNPAG8zft1X38vzn92paxBj+qEq9YmwHjqfkUm7k2PnR+c/wJTBxnWO9O5W4yDt
PmYe5WzPwpdn+q0H/R9OJJc36YLvoEGvczeX/Xz9LPkfgnBDuxcWNdfDOpEN3Wq+
IkHracbebHdIpuS2St4yaElIGmid0M1cu0m4KU5/YhEZ+aBVegF9HI9PLr9Gdbrz
V2PQd8vDobuPL+m+sW5/
=G73u
-----END PGP SIGNATURE-----

#765068#15
Date:
2014-10-15 09:49:14 UTC
From:
To:
Hi Markus,

How about "Domains to avoid error of [wrong number of dots]"?

I'd like to use the "Domains to ..." style like other options.

Any ideas?

Thanks,
--
Tatsuya Kinoshita

#765068#20
Date:
2014-10-15 13:04:57 UTC
From:
To:
Thanks,
--
Tatsuya Kinoshita

#765068#29
Date:
2014-10-15 13:49:44 UTC
From:
To:
Dear Alastair,

Debian Bug Tracking System schrieb am 15. Oct 2014 um 11:24

I'm not able to understand the reason for closing my bugreport because
my report was about w3m in the stable und testing release.

As far as I can see, there is no connection to some paket iapws.

Regards
Markus

#765068#34
Date:
2014-10-15 14:51:27 UTC
From:
To:
Sorry, that's mistake.  I've reopened this bug.

cf.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=765068
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=765058

Thanks,
--
Tatsuya Kinoshita

#765068#39
Date:
2014-10-16 22:53:59 UTC
From:
To:
markus.hiereth@freenet.de wrote:

It took me a while to work out that I wasn't meant to be finding this
in the docs - it's a dialogue constructed out of strings in the source
file "rc.c".

(That really ought to be "use_cookieS", and likewise for most of the
other variable names below.)

<number>?

      If the number of dots in the domain name is less than two, it is
      treated as an invalid cookie (cf. RFC 2109 4.3.2). However, you can

It's not a great variable name, either, because the user isn't being
asked to specify one "wrong" number N which will configure w3m to
reject cookies for domains with N dots - if I understand correctly,
setting N=1 will cause w3m to reject as invalid all cookies for
single-dotted *or* dotless domains (that is, it sets the minimum valid
number of dots to two, which is just begging for off-by-one errors).
It's not quite clear if you can meaningfully set N=255.  And... wait,
why on earth is it a <string>?   What would it mean if I set N=fish?

Tatsuya Kinoshita says
# I'd like to use the "Domains to ..." style like other options.

That would be a good idea if this option was asking for a domain name
string like the preceding options, but if it's really asking for an
integer then it is a different kind of question and there's no point
trying to contort it into the same style.

If we only get to fix the comment string and not the logic, I don't
see any way of saying it that will fit - it would have to be something
like:

  cookie_avoid_wrong_number_of_dots=<string> Reject cookies with this many dots or fewer in their domain
or
  cookie_avoid_wrong_number_of_dots=<string> Number of dots to treat as too few in domains
[...]

I don't see any reason to assume that.

Meanwhile in ja.po it's:

	"[wrong number of dots] を無視するドメイン"

(where the part in Japanese means something like "domains to ignore").
This doesn't seem helpful at all.

#765068#44
Date:
2014-10-17 12:51:39 UTC
From:
To:
Instead, the English messages of the other options should be improved.

If you make a patch to correct the English messages, I'll review it
for merging.

Thanks,
--
Tatsuya Kinoshita

#765068#49
Date:
2014-10-17 19:49:27 UTC
From:
To:
Hello Tatsuya,

please note the discussion thread within the mailing list of the
English translation team:

https://lists.debian.org/debian-l10n-english/2014/10/msg00018.html

The results are

- It is necessary to find out what domain information is subject to
  w3m's checking: The domain of the server that sends a SET-COOKIE
  request and / or the domain name specified in the cookie itself.

- It is necessary to have precisely described what matching is
  performed with the domain attribute of a cookie. E.g. only the
  number of dots in this string or all the conditions mentioned in the
  RFC.

Afterwards, the respective description in the options panel and the
respective paragraph in README.cookies shall be updated.

Best regards
Markus

#765068#54
Date:
2014-10-21 14:29:55 UTC
From:
To:
Hello Tatsuya,

(thanks for informing me that You have passed the German po-file to
the resources for Jessie). Please note with references to:
https://lists.debian.org/debian-l10n-english/2014/10/msg00021.html

In the course of my translation work on the man-page of w3m, I made a
couple of tests and learned to know the option -reqlog

  w3m -reqlog URL

and found out

  ~/.w3m/request.log

as the place for the logged data and what kind of data inside. They
show the HTTP communication between w3m and the web server. The file
features domain name information of both sides as well as the domain
string within a cookie the server tries to set on the disk of the
client PC; in other words, value of its domain attribute of the
cookie. See the snippet below with a SET-COOKIE request by
wikipedia's server and usage of a cookie by w3m.

In addition, the file that stores cookies

  ~/.w3m/cookie

is a plain text file. It is loaded into the browser upon start and
saved before EXIT.

I think with this technical background information, it should be
possible to unravel what domain information is subject to validation
and what are in fact the criteria applied in this validation.

I would appreciate if there was somebody who designs and performs the
necessary tests, The findings would be valuable to improve the w3m's
user documentation.

Regards
Markus
----------------------------------------------------------------------------- GET / HTTP/1.0 User-Agent: w3m/0.5.3+cvs-1.1055 Accept: text/html, text/*;q=0.5, image/*, application/*, audio/*, video/*, message/* Accept-Encoding: gzip, compress, bzip, bzip2, deflate Accept-Language: en;q=1.0 Host: de.wikipedia.org HTTP/1.1 301 Moved Permanently Server: Apache X-Content-Type-Options: nosniff Cache-control: s-maxage=1200, must-revalidate, max-age=0 Last-Modified: Sun, 19 Oct 2014 18:23:54 GMT Location: http://de.wikipedia.org/wiki/Wikipedia:Hauptseite Content-Encoding: gzip Content-Type: text/html; charset=utf-8 Vary: Accept-Encoding,X-Forwarded-Proto,Cookie,X-Use-HHVM X-Varnish: 115743251, 3889558698 3889556238, 2750104885 2749402641 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Content-Length: 20 Accept-Ranges: bytes Date: Sun, 19 Oct 2014 18:34:39 GMT Age: 645 Connection: close X-Cache: cp1055 miss (0), amssq48 hit (25), amssq48 frontend hit (172) X-Analytics: php=zend Set-Cookie: GeoIP=DE:Schwabhausen:48.4000:11.3500:v4; Path=/; Domain=.wikipedia.org GET /wiki/Wikipedia:Hauptseite HTTP/1.0 User-Agent: w3m/0.5.3+cvs-1.1055 Accept: text/html, text/*;q=0.5, image/*, application/*, audio/*, video/*, message/* Accept-Encoding: gzip, compress, bzip, bzip2, deflate Accept-Language: en;q=1.0 Host: de.wikipedia.org Referer: http://de.wikipedia.org/ Cookie: GeoIP=DE:Schwabhausen:48.4000:11.3500:v4 Cookie2: $Version="1" HTTP/1.1 200 OK Server: Apache X-Content-Type-Options: nosniff Content-language: de X-UA-Compatible: IE=Edge Last-Modified: Sun, 19 Oct 2014 10:13:34 GMT Content-Encoding: gzip Content-Type: text/html; charset=UTF-8 Vary: Accept-Encoding,Cookie,X-Use-HHVM X-Varnish: 1111890659, 2301775781 2301775196, 2260380402 2231581926 Via: 1.1 varnish, 1.1 varnish, 1.1 varnish Content-Length: 14505 Accept-Ranges: bytes Date: Sun, 19 Oct 2014 18:34:40 GMT Age: 30065 Connection: close X-Cache: cp1053 miss (0), amssq46 hit (14), amssq49 frontend hit (11335) Cache-Control: private, s-maxage=0, max-age=0, must-revalidate X-Analytics: php=zend
#765068#59
Date:
2023-08-09 13:12:31 UTC
From:
To:
On Fri, 17 Oct 2014 21:49:27 +0200 Markus Hiereth <markus.hiereth@freenet.de> wrote:
[...]

I had a look at the code with a debugger.

The w3m option field 'Domains to avoid [wrong number of dots]' expects a
list of domain names, separated by comma or space.

The code in question is the following from cookie.c:
322  if (version == 0) {
323      /* [NETSCAPE] rule */
324      unsigned int n = total_dot_number(domain->ptr,
325                               domain->ptr + domain->length,
326                               3);
327      if (n < 2) {
328          if (! check_avoid_wrong_number_of_dots_domain(domain)) {
329              COOKIE_ERROR(COO_ESPECIAL);
330          }
331      }

If n < 2 the actual matching happens in file.c:domain_match().

Note that comments in the code talk about RFC 2109 and DRAFT 12 (RFC
2965?). I don't think the code was ever updated to adjust to newer RFCs.
Also note that I'm not really familiar with RFCs related to cookies.

The matching happens against the domain attribute that was given
in the SET-COOKIE header (Domain=).
of the cookie. The version depends of the header name, Set-Cookie: vs
Set-Cookie2: (according to Wikipedia Set-Cookie2 is deprecated and not
used anymore).

The check will only be performed when the number of dots in the domain
name is less then 2. AFAIK RFC 6265 made the leading dot in the domain
attribute optional. This means, a nowadays valid domain attribute, e.g.
github.com, will be checked.

Whitelisting `.github.com' will a match `domain=github.com' while
whitelisting `aol.com' will not match `domain=.aol.com' (.aol.com will
not be checked in the first place because it has two dots. I changed the
code to debug it).

Note, a domain like `https://aol.co.uk' will never be checked as is
always contains at least two dots.