I got the following error when upgrading the "gyp" package. I ran into this from a ubuntu machine but I reproduced it in a minimal sid chroot as well: # dpkg -i gyp_0.1~svn1654-1_all.deb (Reading database ... 12267 files and directories currently installed.) Preparing to unpack gyp_0.1~svn1654-1_all.deb ... Unpacking gyp (0.1~svn1654-1) over (0.1~svn1654-1) ... Setting up gyp (0.1~svn1654-1) ... root@bod:/tmp# dpkg -i gyp_0.1~svn1729-3_all.deb (Reading database ... 12267 files and directories currently installed.) Preparing to unpack gyp_0.1~svn1729-3_all.deb ... Unpacking gyp (0.1~svn1729-3) over (0.1~svn1654-1) ... dpkg: error processing archive gyp_0.1~svn1729-3_all.deb (--install): unable to open '/usr/share/pyshared/gyp-0.1.egg-info/dependency_links.txt.dpkg-new': No such file or directory Errors were encountered while processing: gyp_0.1~svn1729-3_all.deb After some debugging I think the following is what happens: 1. The old package contains a symlink /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info 2. The new package contains a directory /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info 3. On upgrade the symlink is not replaced with the directory, it still is a symlink, this causes the error. I can send a full dpkg debug log. The debs in question can be downloaded from e.g.: http://launchpadlibrarian.net/154612335/gyp_0.1%7Esvn1729-3_all.deb http://launchpadlibrarian.net/145640491/gyp_0.1%7Esvn1654-1_all.deb But that should not be needed, I also created two minimal debs that seem to trigger the issue: $ sudo dpkg -i symlink-becomes-dir_1.0_all.deb && sudo dpkg -i symlink-becomes-dir_2.0_all.deb && echo "have:" && ls -ald /usr/share/symlink-becomes-dir && printf "\nwant:\n" && dpkg -c symlink-becomes-dir_2.0_all.deb |grep /usr/share/symlink-becomes-dir (Reading database ... 436479 files and directories currently installed.) Removing symlink-becomes-dir ... Selecting previously unselected package symlink-becomes-dir. (Reading database ... 436479 files and directories currently installed.) Unpacking symlink-becomes-dir (from symlink-becomes-dir_1.0_all.deb) ... Setting up symlink-becomes-dir (1.0) ... (Reading database ... 436481 files and directories currently installed.) Preparing to replace symlink-becomes-dir 1.0 (using symlink-becomes-dir_2.0_all.deb) ... Unpacking replacement symlink-becomes-dir ... Setting up symlink-becomes-dir (2.0) ... have: lrwxrwxrwx 1 root root 3 Mar 18 10:07 /usr/share/symlink-becomes-dir -> foo want: drwxr-xr-x root/root 0 2014-03-18 10:08 ./usr/share/symlink-becomes-dir/ As for the real gyp deb: When I inspect the system before dpkg cleans up again, I see the following: # ls -al /usr/share/pyshared/gyp-0.1.egg-info total 16 drwxr-xr-x 2 root root 4096 Mar 18 08:46 . drwxr-xr-x 4 root root 4096 Mar 18 08:46 .. lrwxrwxrwx 1 root root 52 Oct 22 02:42 PKG-INFO.dpkg-new -> ../../../../share/pyshared/gyp-0.1.egg-info/PKG-INFO lrwxrwxrwx 1 root root 55 Oct 22 02:42 SOURCES.txt.dpkg-new -> ../../../../share/pyshared/gyp-0.1.egg-info/SOURCES.txt lrwxrwxrwx 1 root root 64 Oct 22 02:42 dependency_links.txt.dpkg-new -> ../../../../share/pyshared/gyp-0.1.egg-info/dependency_links.txt lrwxrwxrwx 1 root root 60 Oct 22 02:42 entry_points.txt.dpkg-new -> ../../../../share/pyshared/gyp-0.1.egg-info/entry_points.txt lrwxrwxrwx 1 root root 57 Oct 22 02:42 top_level.txt.dpkg-new -> ../../../../share/pyshared/gyp-0.1.egg-info/top_level.txt Which looks incorrect, a properly installed deb has no symlinks here: # ls -la /usr/share/pyshared/gyp-0.1.egg-info/ total 28 drwxr-xr-x 2 root root 4096 Mar 18 08:47 . drwxr-xr-x 4 root root 4096 Mar 18 08:47 .. -rw-r--r-- 1 root root 242 Oct 22 02:42 PKG-INFO -rw-r--r-- 1 root root 1145 Oct 22 02:42 SOURCES.txt -rw-r--r-- 1 root root 1 Oct 22 02:42 dependency_links.txt -rw-r--r-- 1 root root 41 Oct 22 02:42 entry_points.txt -rw-r--r-- 1 root root 4 Oct 22 02:42 top_level.txt That is because: # ls -la /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info lrwxrwxrwx 1 root root 40 Jul 23 2013 /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info -> ../../../share/pyshared/gyp-0.1.egg-info Which is a directory on a correctly installed package: # ls -lad /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info/ drwxr-xr-x 2 root root 4096 Mar 18 08:47 /usr/lib/python2.7/dist-packages/gyp-0.1.egg-info/ This is also reported in launchpad as https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/1246730 Thanks, Michael
Hi! […] This is “expected” behaviour, I documented it in the dpkg FAQ because it seems to trip people over. The Debian policy also documents this (§6.6.4). <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Will_dpkg_replace_a_symlink_with_a_directory_or_vice_versa.3F> Given the above, I'll be closing this report if there's no other issues besides this one. Thanks, Guillem
Hi Guillem, [..] Thanks for your reply and the link to the FAQ and sorry for this unneeded bugreport. I should have been move careful in my research. Feel free to close the report. Given that this behavior is not that (that) intuitive I wonder if dpkg could issue a warning during unpack? Maybe something like the attached patch? I'm happy to improve the warning message and add one for the other way around (dir is replaced with symlink). Could you elaborate a little bit what the FAQ means with metadata tracking? Would the $pkg.lists file need to be amended with information like: /path/to/dir =d /path/to/file =f /path/to/symlink =s ? Thanks for your consideration. Cheers, Michael
Hi! No problem. Merging it now with previous incarnations of this. The problem is that this has been the intended behaviour for dpkg all along, mainly to support local admins switching directories to symlinks and the reverse to be able to rearrange filesystems and disks w/o needing whole reinstalls and similar. So starting to issue warnings would be quite annoying in those cases. The issue here is that dpkg does not have enough information to distinguish if the discrepancy is due to a switch in the packaging or due to sysadmin actions. More or less, in addition to file size, uid/gid, mode, symlink target, device IDs, etc. Reusing the .list files is probably not a good idea though, because unfortunately other programs try to use those internal files and they might silently break then. And if those are to disappear, then I might as well try to store the data in a way that makes it faster to load, as .list files are the currently slowest part of dpkg startup, in many situations. My current thinking is to use a new mtree(5) formatted file to track those and also the file checksums. I think I might have the beginning of some code for that somewhere, but I'm not sure. In any case, I still need to consider how to internally handle shared paths with different metadata, like directories or ref-counted paths in M-A:same packages, and metadata switched for those shared paths, for example. Thanks, Guillem