#162283 wget: timstamping for retrieved symlinks unnecessarily downloads files

Package:
wget
Source:
wget
Description:
retrieves files from the web
Submitter:
Jeff Sheinberg
Date:
2015-12-11 09:48:06 UTC
Severity:
normal
#162283#5
Date:
2002-09-25 12:54:05 UTC
From:
To:
Package: wget
Version: 1.8.2-5
Severity: normal

Hi,

Wget always downloads retrieved symlinks, even when the retrieved
file has already been successfully downloaded.

I invoked wget like this,

    wget \
              --cut-dirs=2 \
              --glob=on \
              --input-file=NetBSD-1.6-urlist \
              --append-output=NetBSD-1.6-log \
              --retr-symlinks \
              --timestamping \
              --force-directories \
              --no-host-directories \
              --non-verbose \
            &

the files in question have this ftp directory listing,

    lrwxrwxrwx   1 ftpuser  ftpusers 27 Sep 12 03:35 man.tgz -> ../../../shared/ALL/man.tgz
    lrwxrwxrwx   1 ftpuser  ftpusers 28 Sep 12 03:35 misc.tgz -> ../../../shared/ALL/misc.tgz

I ran wget and these two files, {man,misc}.tgz, were downloaded
successfully, ie, the symlinks were followed, and the
linked-to-files were downloaded, in addition, during this run many
other regular, ie, not symlinked files, were also downloaded.

The first indication of a problem is that the downloaded symlinked
files were given the mtime of the symlink, rather than the mtime
of the linked-to-file,

    -rw-r--r--    1 jeff     jeff 6640184 Sep 12 03:35 man.tgz
    -rw-r--r--    1 jeff     jeff 2333002 Sep 12 03:35 misc.tgz


Then, I ran wget again, and here is where the problem occurred.
What happened is that the previously downloaded regular files were
not downloaded again, as expected, however, the two symlinked
files were downloaded again, in their entirety.

This is a tremendous waste of resources, and it negates the whole
purpose for using timestamping in the first place.

Thanks,

#162283#14
Date:
2003-04-20 23:42:54 UTC
From:
To:
Dear Debian "wget" bug reporter,

I just want to inform you why your bug report is still open or why it
looks like that nobody is working on fixing the bugs reported against
the wget package (http://bugs.debian.org/wget).

Since some month I didn't report bugs and problems to the upstream
author/mailinglist (http://www.gnu.org/software/wget/#mailinglists)
because wget right now has no maintainer who is working on it or fixes
bugs.:(

http://www.gnu.org/help/help.html
--8<--
We are looking for new maintainers for these GNU packages (contact
<maintainers@gnu.org> if you'd like to volunteer):
      * ...
      * wget (which still has a maintainer, but he would like to step
        down)
--8<--

This is the reason why reporting bugs to the wget mailinglist right now
doesn't make sense because nobody will work on them. When there will be
a new maintainer I will report the open bugs in the Debian Bug Tracking
System (BTS) to him but this may take some time (the help request on the
gnu.org page is there since a month; maybe you know somebody:)).

Thanks for reporting bugs.:)

#162283#17
Date:
2003-04-20 23:42:54 UTC
From:
To:
Dear Debian "wget" bug reporter,

I just want to inform you why your bug report is still open or why it
looks like that nobody is working on fixing the bugs reported against
the wget package (http://bugs.debian.org/wget).

Since some month I didn't report bugs and problems to the upstream
author/mailinglist (http://www.gnu.org/software/wget/#mailinglists)
because wget right now has no maintainer who is working on it or fixes
bugs.:(

http://www.gnu.org/help/help.html
--8<--
We are looking for new maintainers for these GNU packages (contact
<maintainers@gnu.org> if you'd like to volunteer):
      * ...
      * wget (which still has a maintainer, but he would like to step
        down)
--8<--

This is the reason why reporting bugs to the wget mailinglist right now
doesn't make sense because nobody will work on them. When there will be
a new maintainer I will report the open bugs in the Debian Bug Tracking
System (BTS) to him but this may take some time (the help request on the
gnu.org page is there since a month; maybe you know somebody:)).

Thanks for reporting bugs.:)

#162283#22
Date:
2007-07-18 18:30:49 UTC
From:
To:
tags 162283 + moreinfo
thanks

Hello Jeff,

I'm working on reproducing all the debian wget bugs and then report them
upstream. Can you still reproduce your reported bug
http://bugs.debian.org/162283 with wget 1.10.2 (in etch/stable and
later?
Because the input-file "NetBSD-1.6-urlist" is not in the report, I'm
asking before trying out myself.:)

thx.

#162283#29
Date:
2007-07-18 19:49:58 UTC
From:
To:
tags 162283 + upstream
forwarded 162283 https://savannah.gnu.org/bugs/index.php?20522
thanks

I have not attempted to reproduce this bug; however, if the behavior is
still present in wget, I would wish to address it. Therefore, to make
sure I don't forget about it, I've added it to our upstream bug tracker:
https://savannah.gnu.org/bugs/index.php?20522

#162283#36
Date:
2015-09-21 09:12:57 UTC
From:
To:
VOUS ÊTES PLUTÔT MAS EN PIERRES OU VILLA CONTEMPORAINE?

Madame, Monsieur, accédez à un cadre de vie authentique, aux senteurs
de la Provence, avec OPUS Développement. Découvrez nos derniers mas en
pierres et villas contemporaines, ici!


Vous êtes abonné à la newsletter d'OPUS Développement avec l'adresse
email: 162283-submitter@bugs.debian.org

Vous pouvez vous désinscrire: http://communication.villas-lumina.com/HD?b=QuHHs4jMP7AxKoMH1zhHPhyQG2iGjFiOkIahzp7rN76AbU4rnSQUOV5TCCjPBx-8&c=ZjBqNUrfTOxMHJ5mE8Oiaw des offres d'OPUS Développement.

En application de la loi n°78 - 17 du 6 Janvier 1978 modifiée par la
loi du 6 Août 2004 relative à l'informatique, aux fichiers et aux
libertés, vous disposez d'un droit d'accès, de modification, de
rectification et de suppression des données personnelles vous
concernant auprès de:
OPUS Développement 4, rue des Trésoriers de la Bourse 34000 Montpellier.

#162283#41
Date:
2015-10-12 09:50:18 UTC
From:
To:
VOUS ÊTES PLUTÔT MAS EN PIERRES OU VILLA CONTEMPORAINE?

Madame, Monsieur, accédez à un cadre de vie authentique, aux senteurs
de la Provence, avec OPUS Développement. Découvrez nos derniers mas en
pierres et villas contemporaines, ici!


Vous êtes abonné à la newsletter d'OPUS Développement avec l'adresse
email: 162283-quiet@bugs.debian.org

Vous pouvez vous désinscrire: http://communication.villas-lumina.com/HD?b=A8zFmVHos5oxYiLli5Tgvv9OZHO3u00rw9TVCMWCHhmPMQInXj0pZFcGrhWE2N7b&c=LrvdRUUeQuUJPf_jFbimIw des offres d'OPUS Développement.

En application de la loi n°78 - 17 du 6 Janvier 1978 modifiée par la
loi du 6 Août 2004 relative à l'informatique, aux fichiers et aux
libertés, vous disposez d'un droit d'accès, de modification, de
rectification et de suppression des données personnelles vous
concernant auprès de:
OPUS Développement 4, rue des Trésoriers de la Bourse 34000 Montpellier.

#162283#44
Date:
2015-12-11 09:46:19 UTC
From:
To:
Pour visualiser ce message sur votre navigateur:


VOUS ÊTES PLUTÔT MAS EN PIERRES OU VILLA CONTEMPORAINE?

Madame, Monsieur, accédez à un cadre de vie authentique, aux senteurs
de la Provence, avec OPUS Développement. Découvrez nos derniers mas en
pierres et villas contemporaines, ici!


Vous êtes abonné à la newsletter d'OPUS Développement avec l'adresse
email: 162283-submitter@bugs.debian.org

Vous pouvez vous désinscrire: http://communication.villas-lumina.com/HD?b=YE8bWXbPSeegWbgXDesc1CC9tJab5SVzaHHLayUm4ppkaGydchXaeqK4n3BUGevH&c=URxNOz6iecUOdtreOtJ7ww des offres d'OPUS Développement.

En application de la loi n°78 - 17 du 6 Janvier 1978 modifiée par la
loi du 6 Août 2004 relative à l'informatique, aux fichiers et aux
libertés, vous disposez d'un droit d'accès, de modification, de
rectification et de suppression des données personnelles vous
concernant auprès de:
OPUS Développement 4, rue des Trésoriers de la Bourse 34000 Montpellier.