As with some other d-i reports, the Version is set from the ISO image,
as I'm not sure how to get the d-i version proper from that.

My current best test case for this:

  (blkid lines are split for readability.)

  1. The starting conditions: I've pared them down to two small disks,
     tested in a QEMU+KVM virtual machine.  Both have GPT partition
     tables, per [[STARTING-TABLES]] below.  Take note of disk 2,
     partition 3, which is marked with an LVM type code (applied via
     setting 8e00 in gdisk) but in fact contains a LUKS volume which
     contains an LVM PV.

     (gdisk doesn't have an obvious type code for "Linux LUKS volume",
     and a straw poll of another Linux sysadmin says they do the same
     thing I do and use the underlying type.  Maybe "Linux reserved"
     would be more accurate?)

     host# losetup --show --find disk2
     /dev/loop0
     host# kpartx -a /dev/loop0
     host# blkid /dev/mapper/loop0p3
     /dev/mapper/loop0p3:
     UUID="67e3b0d0-986d-43d3-9618-f6f8895ebbb7"
     TYPE="crypto_LUKS"
     PARTLABEL="Linux LVM"
     PARTUUID="1eec58ea-5479-49f1-88cb-308dd583ff66"

  2. Boot the Debian 7.6.0 amd64 installer:

     host% /sbin/blkid /dev/cdrom
     /dev/cdrom:
     UUID="2014-07-12-14-23-02-00"
     LABEL="Debian 7.6.0 amd64 1"
     TYPE="iso9660"
     PTUUID="46bf9aa7"
     PTTYPE="dos"

     host% qemu-system-x86_64 -enable-kvm \
         -cdrom /dev/cdrom -boot d \
	 -hda disk1 -hda disk2 -m 4096 -monitor stdio

  3. Choose all defaults (and meaningless usernames, etc.) up until
     the partitioning stage.  Then choose "Manual" partitioning.

     At this point it will be apparent to the particularly alert
     viewer that both sda3 and sdb3 are shown with a "K" and an "lvm"
     marker.  (I'm not sure what the "K" means; maybe it's meant to
     represent a flaming skull?)

     From the second console:

     virt# blkid /dev/sdb3 # (line split)
     /dev/sdb3:
     UUID="67e3b0d0-986d-43d3-9618-f6f8895ebbb7"
     TYPE="crypto_LUKS"

     ... so blkid from the d-i environment at this stage at least
     recognizes that there is a typed volume on that block device
     (this seems to be true earlier on as well).

  4a. Choose sda2, and set it to Ext4, mount point /boot, then "Done"
      to return to "Partition disks".

  4b. Choose sda3, and set it to "physical volume for encryption",
      "Erase data: no", then "Done" to return to "Partition disks".

  5. Choose "Configure encrypted volumes".  A dialog about which
     changes will be made to the disks appears.  The main listings
     read:

     | The partition tables of the following devices are changed:
     |   SCSI1 (0,0,0) (sda)
     |
     | The following partitions are going to be formatted:
     |   partition #2 of SCSI1 (0,0,0) (sda) as ext4

     Choose "Yes" to write changes to disks.

  6. Choose "Create encrypted volumes", then select /dev/sda3 only.
     "Continue", then "Finish".  Enter an arbitrary passphrase.
     Some progress bars appear, then "Partition disks" again.

  7. Choose sda3_crypt part 1, and set it to "physical volume for LVM",
     then "Done" to return to "Partition disks".

  8. Choose "Configure the Logical Volume Manager".  The alert viewer
     will notice that there are more "Free Physical Volumes" than
     there should be.

     At this point sdb3 has _already_ been reinitialized as a new PV,
     destroying its LUKS header and rendering it unrecoverable except
     by restoring from backup:

     host# blkid /dev/mapper/loop0p3 # (line split)
     /dev/mapper/loop0p3:
     UUID="1jZyPo-mXqX-GuSt-L1KP-knSn-j6Yw-5yl2ct"
     TYPE="LVM2_member"
     PARTLABEL="Linux LVM"
     PARTUUID="1eec58ea-5479-49f1-88cb-308dd583ff66"
     host# cryptsetup luksOpen /dev/mapper/loop0p3 DITest_pv
     Device /dev/mapper/loop0p3 is not a valid LUKS device.

Note especially that none of the warning screens that normally appear
to confirm which partitions will have their data destroyed refer to
sdb3 at all, and this occurs regardless of whether I create any VGs
incorporating sdb3 as a PV.  Continuing with the installation in this
vein, if I:

  - create a single VG using only sda3_crypt,

    + for which the selection dialog for PVs displays _all_ available
      block devices, not merely those marked for LVM use per se---so
      the idea that sdb3 is now a PV is not made obvious by that means
      (but this would be too late anyway)

  - then a single LV on it with an ext4 root partition

then a warning appears about overwriting data on the VG and the LV,
but makes no reference to the physical partitions.

If I go for "Configure the Logical Volume Manager" _first_, there is
some kind of warning about not being able to change the partition
tables of the disks on which PVs will be placed later, which I didn't
investigate further, but this doesn't appear in the above sequence.

The original configuration was somewhat more complicated than this
test case, which also made it harder to see the non-target disks as
most of them were off the screen.  I also used EFI boot into an
LXDE-variant expert install then; I don't think that matters here.

Outcome: all attached LVM+LUKS+GPT volumes were destroyed.  :-( :-(
:-(

Expected outcome: "Manual" partitioning mode should only ever
overwrite data on volumes specifically designated by the user.
Additionally, I would normally expect that:

  - Partitions with _existing_ LVM type codes but no recognizable PV
    header should not be presumed to be uninitialized PVs without
    asking the user.  (What if a future LVM release creates a new,
    incompatible PV type, even, and the user wants to incorporate
    the existing volume?)

    + ... and _certainly_ not if they have a header recognizable by
      blkid, which might apply more generally too.

  - The warning screen used for writing new partition tables and
    filesystems should also appear before physically initializing LVM
    PVs, LUKS, etc., as that would be the clearest for the user to
    know "which data might I be about to vaporize" and have the option
    to back out.

I rechecked the Installation Guide and Release Notes and I didn't see
anything about this specifically, but I'd sure appreciate a pointer if
I just missed it somehow.

Unrelatedly, I was actually planning on unplugging all the non-target
disks first as a precautionary measure, but then I forgot to and
didn't think anything further of it until the cold chill of cryptsetup
failing when I tried to read anything from them.

Now I am sad and have filesystems to reconstruct.  I had backups of
the more important unreplaceable stuff, but some of the configuration
will be a major pain.  :-(

(One might say the real lesson is "never install with insufficient
sleep and insufficient tea", but anyway.)

I'll upload the test disk images shortly.

disk1_pristine, disk2_pristine, and the key file for the test LUKS volume on p3 of
disk2 are now available at:

http://dasyatidae.net/tmp/d-i-test-20141109.tar.xz (49 MiB)
  SHA-256: f35d2a8573998f6a82949cf8bb302b1e34c7fdc3f184545f8d682a6e19bb1059

Use tar -S or equivalent to unpack!  The disk images are raw, 4 GiB sparse files.
And please make a local copy of this file if you wish to keep it.

The LUKS volume should contain an LVM PV in a single-PV VG, with one LV with an ext4
filesystem with one test file in it, though most of that is not directly relevant to
the report.

It seems that (debugging with the assumption that latest git and the on-DVD version
are Sufficiently Similar):

  1. Flags for partitions are retrieved via libparted, which sets the "lvm" flag
     on any partitions that have that GPT type.
  2. partman-lvm/init.d/lvm sets the "lvm" method on all partitions with the
     "lvm" flag, resulting in (3) from the test sequence.
  3. partman-lvm/lib/lvm-base.sh:pv_list assumes all devices with the method "lvm"
     are PVs.
  4. partman-lvm/choose_partition/lvm/do_option:do_initial_setup iterates over
     pv_list and does pv_create on everything in it immediately.
  5. partman-lvm/lib/lvm-base.sh:pv_create tests for the presence of an active
     PV using the exit code of the pvs command, and anything that doesn't indicate
     one results in pvcreate -ff, resulting in (8) from the test sequence.  Running
     pvcreate without -f interactively will detect the LUKS signature and offer an
     abort, as it happens.

On the immediate front, it _may_ be that testing whether it's "safe" to create a
new PV there would be better done via something like

  pvcreate --test --quiet -- "$pv" </dev/null >/dev/null 2>&1

(which e.g. exits 5 on finding a LUKS signature) rather than checking using ! pvs.
Or, better yet, along the lines of crypttab(5)'s precheck=un_blkid.  By itself, that
would probably make the install fail without further reworking though.

A better question would probably be why that flag->method propagation is done in
manual partitioning mode in the first place; I assume that's used for something,
but I don't know what, having not delved into this code before.

as a type code for LUKS are also affected by this.  So it's not just GPT.
It's arguably even more dangerous for MBR, because the type code space is
so small that collisions should be expected, but util-linux's fdisk in MBR
mode also provides a 0xda code for "non-FS data", so users in that case may
be less tempted to default to the underlying volume type.

Hi Drake,

I've just reproduced your findings here, and I'm looking at the code
right now. As you've guessed, the partman-lvm code currently
unconditionally tries to set up *every* partition with an LVM
partition type, regardless. If you're interested the code is in
partman-lvm/choose_partition/lvm/do_option:do_initial_setup(). It
calls into partman-lvm/lib/lvm-base.sh:pv_create(), and pv_create()
checks to see if the partition is already set up as a PV (by calling
"pvs") - if so, it leaves it alone, otherwise it calls "pvcreate".

So... There are a few things to do here:

 1. Don't do what you're doing! This is one of the few areas where the
    partition type matters in d-i
 2. I'm looking to add a check in pv_list() so it will either:
    (a) Ignore partitions tagged with LVM type but some other
        filesystem/blkid contents; OR
    (b) Warn about such partitions and ask the user what to do.

2(a) looks much easier, I'll be honest, so that's my plan for now.

As a point of information, you weren't sure what the "K" tag means in
the d-i partitioner. Ironically, it's short for "Keep", i.e. don't
touch this partition...

Control: severity -1 important
Control: clone -1 -2
Control: retitle -2 Installation manual should warn about the use of LVM partition types
Control: reassign -2 installation-guide
defeat I'm afraid. The code in partman-lvm is very flexible in terms
of allowing user choice, but that actually makes this particular case
even harder.

The best thing I can tell you is: don't use the partition types for
LVM unless you really want to use LVM on those partitions! Sorry. :-/

We should add a warning in the installation manual for this corner
case, at least. I've opened a new bug for that above.