#930796 spindown_time and force_spindown_time are broken in hdparm 9.58+ds-1

Package:
hdparm
Source:
hdparm
Description:
tune hard disk parameters for high performance
Submitter:
Sébastien Béhuret
Date:
2019-10-10 16:39:15 UTC
Severity:
important
#930796#5
Date:
2019-06-20 18:42:17 UTC
From:
To:
Dear Maintainers,

In this version of hdparm, a new option 'force_spindown_time' was
introduced to set the spindown time for disks that don't support APM.
This option is supposed to translate to hdparm -S, similarly to the
original option 'spindown_time'.

hdparm package comes with 3 main scripts:

1) /usr/lib/pm-utils/power.d/95hdparm-apm
This script will translate 'force_spindown_time' to hdparm -S and apply the
option even if APM was not detected.
This is the desired behavior.

2) /etc/apm/event.d/20hdparm
This script will ignore /etc/hdparm.conf and apply hard-coded defaults
instead.
This behavior is unexpected.
Expected/Desired behavior: Read /etc/hdparm.conf and apply relevant options.

3) /lib/hdparm/hdparm-functions (sourced from /lib/udev/hdparm, which is
invoked by udev rule /lib/udev/rules.d/85-hdparm.rules)
- 'force_spindown_time' is buggy because it is not converted back to -S,
which leads to a syntax error during hdparm execution (e.g. hdparm
force_spindown_time$VALUE instead of hdparm -S$VALUE).
- Both options 'spindown_time' and 'force_spindown_time' are processed even
if APM is not supported. From the comments in the configuration file
(/etc/hdparm.conf), it is understood that 'spindown_time' will be applied
for APM disks only and 'force_spindown_time' for all disks (or possibly for
non-APM disks only).
- The scripts will also apply hard-coded defaults for -S and -B if APM was
detected. The hard-coded defaults differ from those used in
/etc/apm/event.d/20hdparm, leading to inconsistent behavior.

4) Additional issues with non-APM disks:
- Manually invoking hdparm -S$VALUE /dev/sdx is simply ignored even though
hdparm executes successfully. The disks do not spin down after the time
delay when there was no access.
- Manually invoking hdparm -y /dev/sdx will spin down the disks
immediately. The disks will not wake up unless they are accessed, which is
the expected behavior.

These were all working fine in hdparm 9.51+ds-1+deb9u1, which is the
current version in stretch.

In short, it is currently impossible to obtain a consistent and working
configuration for non-APM disks.

Many thanks and regards,
Sebastien Behuret

#930796#10
Date:
2019-06-24 15:54:05 UTC
From:
To:
Hi Sebastien,

2. As APM is almost dead and most likely there are no laptops using APM
and buster. I'll drop /etc/apm/event.d/20hdparm in the next release.

3. This is a real issue. In /lib/hdparm/hdparm-functions I've left the
"force_spindown_time$VALUE" option intentionally, it need to be
translated to "-S" later in scripts using hdparm-functions like it is
done in 95hdparm-apm

/lib/udev/hdparm is called by udev and need to be fixed.

/usr/lib/pm-utils/power.d/95hdparm-apm called by pm-utils events and
takes care only about spin_down and apm options for the disks which
support apm.

To obtain a consistent behavior /lib/udev/hdparm can call
/usr/lib/pm-utils/power.d/95hdparm-apm for spindown and apm options and
hdparm directly for all other options.

4. I failed to reproduce that. I couldn't put to standby a non-apm disk
on a stretch system with hdparm -S (hdparm 9.51)
Could you please try to build hdparm 9.51 or just get a binary package
and run it to see if 9.51 works for your disks compared to 9.58?

Thank you for the detailed report.
Alex

#930796#17
Date:
2019-06-27 13:25:25 UTC
From:
To:
Hi Alex,

Thanks for your detailed reply.

2. I agree that it is appropriate to drop /etc/apm/event.d/20hdparm.

3. Your solution is OK: Calling /usr/lib/pm-utils/power.d/95hdparm-apm from
/lib/udev/hdparm would fix the force_spindown_time conversion issue.

With your solution I assume that /lib/udev/hdparm would call hdparm twice
on each HDD during udev invocation, once for non-spindown options returned
by /lib/hdparm/hdparm-functions, and once through
/usr/lib/pm-utils/power.d/95hdparm-apm for spindown options. Custom scripts
relying on hdparm_options() function in /lib/hdparm/hdparm-functions would
still fail if force_spindown_time is used in /etc/hdparm.conf. I would
suggest implementing the conversion code directly into hdparm_options()
function to avoid code duplication, prevent misuse, and possibly avoid
calling hdparm twice on each HDD.

4. Thanks for your feedback. I have done some experiments and it appears
that the -S issue comes from something else. I can only confirm that the -S
option was still working fine at the time of hdparm 9.56+ds-2 in
buster/testing (Fall 2018) and it had been working for over 5 years with
various kernel and hdparm versions. Between hdparm 9.56+ds-2 and hdparm
9.58+ds-1, the kernel was updated (4.17.8-1 => 4.19.37-3) and there were
also changes in udev (239-7 => 241-3).

Below is a summary of what I did so far to try and debug hdparm -S:


A) hdparm versions tried:

$ hdparm-jessie -V
hdparm-jessie v9.43

$ hdparm-stretch -V
hdparm-stretch v9.51

$ hdparm-buster -V
hdparm-buster v9.58


B) What currently works for all versions:

$ hdparm -y /dev/sdx

/dev/sdx:
 issuing standby command

$ hdparm -C /dev/sdx

/dev/sdx:
 drive state is:  standby

## Accessing a mounted partition on /dev/sdx ##

$ hdparm -C /dev/sdx

/dev/sdx:
 drive state is:  active/idle

## Will still work if hdparm -y is repeated at this stage ##


C) What worked before at the time of hdparm 9.56+ds-2 (successful spindown
after the delay):

$ hdparm -S248 /dev/sdx

/dev/sdx:
 setting standby to 248 (4 hours)

## Other delays not tested ##


D) What does not work (anymore) for all versions (hdparm runs successfully
but will not spindown after the delay):

$ hdparm -S1 /dev/sdx

/dev/sdx:
 setting standby to 1 (5 seconds)

$ hdparm -S10 /dev/sdx

/dev/sdx:
 setting standby to 10 (50 seconds)

$ hdparm -S241 /dev/sdx

/dev/sdx:
 setting standby to 241 (30 minutes)

$ hdparm -S248 /dev/sdx

/dev/sdx:
 setting standby to 248 (4 hours)


Best regards,
Sebastien

On Mon, Jun 24, 2019 at 4:54 PM Alex Mestiashvili <amestia@rsh2.donotuse.de> wrote:

#930796#22
Date:
2019-06-28 09:40:13 UTC
From:
To:
/lib/udev/hdparm will call "/usr/lib/pm-utils/power.d/95hdparm-apm"
For the other option hdparm will be called the second time. But I see no
problem here. Please see the updated script here:
https://salsa.debian.org/debian/hdparm/blob/930796/debian/udev-scripts/hdparm

With the new /lib/udev/hdparm, hdparm follows the logic below:

No config (/etc/hdparm.conf doesn't list any drives):
  * If disk supports APM, the defaults:
    - on boot, -B 254
    - on power, -B 254
    - on battery -B 128 -S36 (3 min)
  * no APM support:
    - hdparm will not run (no config!)

If disk config is present in /etc/hdparm.conf:
  * disk supports APM
    - on boot, udev will call /lib/udev/hdparm, which in turn will call
      /usr/lib/pm-utils/power.d/95hdparm-apm for apm options and hdparm
      for other options.
    - on power, /usr/lib/pm-utils/power.d/95hdparm-apm
    - on battery, defaults -B 128 -S 36, use apm_battery and
      spindown_time to set non-default values
  * no APM support:
    - force_spindown_time and other options are applied,
      apm and spindown_time are ignored

For USB or FireWere disks, APM & spindown_time options are ignored,
other options are applied, force_spindown_time will be applied too.
There is bug, https://bugs.launchpad.net/bugs/515023
explaining why USB and FireWire drives are ignored, however the
situation might have improved since then.

This makes sense, but
1. hdparm-functions is the debian specific helper script. The chances
that somebody will use it for custom scripts is very low.
2. force_spindown_time is a hackish workaround and in order to implement
it I need to parse this option later in "95hdparm-apm" script.
Implementing proper handling of "force_spindown_time" in
hdparm-functions will result in bringing part of
resume_hdparm_spindown() function from 95hdparm-apm in hdparm-functions
code. I don't like this idea, but please feel free to implement and send
me a patch. :)

To exclude hdparm, one can try to build hdparm 9.58 on a stretch system.
Building it with make will also work.

Good, from my expirience -y works reliable most of the time, however
some disks may wakeup when hdparm -C is invoked, one can use
smartctl -i -n standby $disk instead.
First of all the timeout -S248 is really long. As far as I know some
disks may decide to go for standby by themselves. Are you sure that
during this 4 hours there is no disk activity? Can you see that the
drive goes to standby exactly after 4 hours and not 2.5?

In hdparm diff between 9.56 and 9.58 I do not see APM related changes:
https://salsa.debian.org/debian/hdparm/compare/upstream%2F9.56+ds...upstream%2F9.58+ds

I also see no reason why a drive will go to standby in 4 hours if it
doesn't work in 5 seconds with -S1 ?

There is also another thing to consider. For some disks you need to set
apm below 128 in order to make them accept spindown.
In general apm (-B) values from 1 to 127 permit spindown, whereas values
from 128 to 254 do not. 255 suppose to disable APM and spindown.

Alternative solution for spindown is hd-idle which is not yet in buster
but is available on salsa: https://salsa.debian.org/debian/hd-idle

Best,
Alex

#930796#27
Date:
2019-06-29 14:15:23 UTC
From:
To:
The updated script looks just right with -B and -S options going through
95hdparm-apm and other options applied locally.

Thanks!

This is great default behavior. Calling 95hdparm-apm from /lib/udev/hdparm
also prevents setting options that laptop-mode-tools would normally handle.

I was unaware of this bug and never experienced this issue with external
USB drives. I do remember external USB drives going into standby mode
shortly after backup completion, but this does not occur anymore in debian
buster/testing. The drives in question do not support APM so it makes sense
given that -S36 is no longer applied in this case.

The logic that you described above is just fine and you are absolutely
right that it is unlikely hdparm-functions is/will be used for custom
scripts. If the force_spindown_time hack was implemented in
hdparm-functions, it would also be necessary to detect laptop-mode-tools
and parse its configuration there, making things a little trickier.


You are right, unfortunately I won't be able to make this test now.

I'm confident that hdparm -S is somehow broken is recent buster debian due
to:
- Multiple drives affected by the issue (internal and USB external)
- These drives will spin down successfully with hdparm -y, and will stay in
standby mode unless manually accessed (tested for over 48 hours)
- hdparm -S runs successfully but none of the delays work (tested delays
ranged from a few seconds to a few hours)
- It had been working flawlessly with this hardware running debian testing
up to Fall 2018 (hdparm 9.56, kernel 4.17, udev 239)

Using previous versions of hdparm (which used to work) did not solve the
issue, so it is unlikely that hdparm is the actual cause.

I've never timed the actual spindown delay, hdparm states the 'human delay'
after issuing -S (e.g. -S248 gives 4 hours) and it has always seemed right
to me. It is true that the actual delay may be vendor-specific, and some
drives may not support all delays. The disks will not wake up after
invoking hdparm -y manually, thus eliminating the possibility that system
noise prevented automatic spindown.

Some drives may not support short delays IIRC.

There is also another thing to consider. For some disks you need to set

Not sure about that. In previous versions, hdparm would attempt to set
-B254 (128 on battery) on all drives but would fail if APM was not
supported.

Do you know if there are any other options that could affect automatic
spindown?

Alternative solution for spindown is hd-idle which is not yet in buster

Thank you for suggesting hd-idle.

At this stage it would be great if somebody could confirm if they
successfully achieved automatic spindown (hdparm -S, spindown_time,
force_spindown_time) in a recent buster/testing system.


Best regards,
Sebastien

#930796#32
Date:
2019-07-01 11:23:21 UTC
From:
To:
not getting any hdparm settings (not event -S36) as they were not used on
battery mode and spindown feature is disabled by default for USB drives in
recent hdparm versions. This must have been an internal feature from WD as
documented here: https://support-en.wd.com/app/answers/detail/a_id/16047

The fact that automatic spindown does not work anymore for these drives in
buster/testing may indicate that there is some form of system noise (but
somehow this noise would not be sufficient to wake up the drives after
hdparm -y) or that something else is actively disabling automatic spindown.

#930796#37
Date:
2019-10-09 06:58:34 UTC
From:
To:
Hi Sébastien,

I still have no solution for hdparm, but as workaround one can try
hd-idle which is now available via buster-backports or testing.

Best regards,
Alex

#930796#42
Date:
2019-10-10 16:34:51 UTC
From:
To:
Hi Alex,

Apologies for the late reply, I found what prevented the HDDs to enter
standby mode: udisksd. I leave the workarounds here for reference.

Intro: The smartd and udisksd [1] daemons poll S.M.A.R.T. data from drives
regularly, and HDDs with a longer standby (or spindown) timeout than the
polling interval may fail to enter standby. In the case of udisksd, drives
that are already spun down are usually not affected, and standby timeout
applied by udisks2 seems to be unaffected.

Workarounds for smartd:
- Add -i value/--interval=value option to smartd_opts in
/etc/default/smartmontools, using a value greater than the standby timeout.
- Add -n standby or -n standby,q to DEVICESCAN statement in
/etc/smartd.conf to prevent checking disks in standby, and further suppress
log message to that effect so as not to cause a write to disk.

Workaround for udisksd:
- Run systemctl mask udisks2 to prevent udisksd execution.

Other possible workarounds could be setting the standby timeout to a
duration lower than the default polling interval (1800 seconds for smartd,
10 minutes for udisksd), forcing a manual spindown using hdparm -y
/dev/sdx, or trying hd-idle as suggested earlier in this thread.

Best regards,
Sebastien

[1]
https://wiki.archlinux.org/index.php/Udisks#Broken_standby_timer_(udisks2)

On Wed, Oct 9, 2019 at 7:58 AM Alex Mestiashvili <amestia@rsh2.donotuse.de> wrote: