#1076022 Backport some security settings from upstream 3.2.5 release to mitigate BlastRADIUS #1076022
- Package:
- freeradius
- Source:
- freeradius
- Description:
- high-performance and highly configurable RADIUS server
- Submitter:
- Herwin Weststrate
- Date:
- 2024-10-02 12:06:01 UTC
- Severity:
- normal
- Tags:
FreeRADIUS 3.2.5 has just been released, which includes some security fixes for BlastRADIUS: a vulnerability with a name and a website[0] and a logo (hadn't seen one of those in a while). The FreeRADIUS security page[1] (scroll to "2024.07.09", there is no anchor to link directly to the relevant article) describes some new configuration options to resolve everything. Since this will be the first thing people read, it would be nice to have those backported to the Debian packages. At first glance, it looks like this requires just two commits[2] [3] to be cherry-picked, but there may be some hidden dependencies in previous commits. [0] https://www.blastradius.fail/ [1] https://www.freeradius.org/security/ [2] https://github.com/FreeRADIUS/freeradius-server/commit/0947439f2569d2b8c2b4949be24250263934e260 [3] https://github.com/FreeRADIUS/freeradius-server/commit/6616be90346beb6050446bd00c8ed5bca1b8ef29
Am 09.07.24 um 18:15 schrieb Herwin Weststrate: I haven't looked closer yet, but the patches do not apply at all dpkg-source: info: applying 0947439f2569d2b8c2b4949be24250263934e260.patch patching file raddb/radiusd.conf.in Hunk #1 FAILED at 625. Hunk #2 FAILED at 643. 2 out of 2 hunks FAILED patching file src/include/clients.h Hunk #2 FAILED at 52. 1 out of 2 hunks FAILED patching file src/include/libradius.h Hunk #1 FAILED at 411. 1 out of 1 hunk FAILED patching file src/include/radiusd.h Hunk #1 FAILED at 178. Hunk #2 succeeded at 564 (offset -6 lines). 1 out of 2 hunks FAILED patching file src/lib/radius.c Hunk #1 succeeded at 2631 (offset -128 lines). Hunk #2 FAILED at 2770. Hunk #3 FAILED at 2790. 2 out of 3 hunks FAILED patching file src/main/client.c Hunk #1 succeeded at 489 (offset -2 lines). Hunk #2 FAILED at 515. Hunk #3 succeeded at 904 (offset -16 lines). Hunk #4 succeeded at 914 (offset -16 lines). Hunk #5 succeeded at 1173 (offset -30 lines). 1 out of 5 hunks FAILED patching file src/main/listen.c Hunk #1 succeeded at 508 (offset -22 lines). Hunk #2 FAILED at 683. Hunk #3 succeeded at 1763 (offset -271 lines). Hunk #4 FAILED at 2109. Hunk #5 succeeded at 1846 (offset -271 lines). 2 out of 5 hunks FAILED patching file src/main/mainconfig.c Hunk #2 FAILED at 88. Hunk #3 FAILED at 164. Hunk #4 succeeded at 849 (offset -24 lines). Hunk #5 FAILED at 1173. 3 out of 5 hunks FAILED dpkg-source: info: applying 6616be90346beb6050446bd00c8ed5bca1b8ef29.patch patching file raddb/clients.conf Hunk #1 FAILED at 137. Hunk #2 FAILED at 152. 2 out of 2 hunks FAILED patching file raddb/proxy.conf Hunk #1 FAILED at 255. 1 out of 1 hunk FAILED patching file raddb/radiusd.conf.in Hunk #1 FAILED at 604. Hunk #2 FAILED at 632. Hunk #3 FAILED at 691. 3 out of 3 hunks FAILED patching file src/include/clients.h Reversed (or previously applied) patch detected! Skipping patch. 2 out of 2 hunks ignored patching file src/include/libradius.h Hunk #1 succeeded at 942 (offset -28 lines). patching file src/include/radiusd.h Hunk #1 FAILED at 176. 1 out of 1 hunk FAILED patching file src/include/realms.h Hunk #1 FAILED at 71. 1 out of 1 hunk FAILED patching file src/main/client.c Hunk #1 FAILED at 491. Hunk #2 FAILED at 514. Hunk #3 FAILED at 727. Hunk #4 FAILED at 920. Hunk #5 FAILED at 930. Hunk #6 FAILED at 1203. Hunk #7 succeeded at 1494 (offset -37 lines). 6 out of 7 hunks FAILED patching file src/main/listen.c Hunk #1 FAILED at 532. Hunk #2 FAILED at 543. Hunk #3 FAILED at 683. Hunk #4 FAILED at 2114. Hunk #5 FAILED at 2546. 5 out of 5 hunks FAILED patching file src/main/mainconfig.c Hunk #1 FAILED at 73. Hunk #2 FAILED at 211. Hunk #3 FAILED at 921. Hunk #4 FAILED at 1225. 4 out of 4 hunks FAILED patching file src/main/process.c Hunk #1 FAILED at 2806. Hunk #2 FAILED at 2823. 2 out of 2 hunks FAILED patching file src/main/realms.c Hunk #1 FAILED at 481. Hunk #2 FAILED at 789. 2 out of 2 hunks FAILED Even with fuzz 80% of the hunks do not apply. Given that the freeradius codebase is really complicated I'm not entirely sure whether we can do this (_I_ can't), or ask the security team for a newer upstream version in stable. But I'll give 3.2.5 a go in unstable ASAP. Bernhard
these two commits. Pretty much every commit of July 8 was relevant. I've created a new git repo where I imported the extracted Debian package, added the upstream repository as a new remote, and git cherry-picked every commit of that day except for the changelogs and the CentOS CI updates. The conflicts were all related to missing code that has been added in recent upstream versions and pretty easy to fix. The result is this 2500 line behemoth of `git log -p`. I tested it with the user `bob` enabled (the default test user of FreeRADIUS) and setting `require_message_authenticator = auto` and using a different machine to send requests to it. The `auto` settings looks to be working: if I start with an `Access-Request` with no `Message-Authenticator` attribute, the logging of FreeRADIUS shows this client is vulnerable and further checking will be disabled. After a restart of the server and sending an `Access-Request` with the `Message-Authenticator` attribute the client will have checking enabled, and sending a next request without the attribute will result in the package being dropped. Replies of the server now always include a `Message-Authenticator` attribute, which they did not have before (with the default config). A simple authentication with 802.1X (PEAP) looked like it's still working as well. I have not yet tested the proxy settings, it takes a while to set that up and I would first like to know if there is a chance that this patch set will be accepted, if it gets rejected right away for whatever reason I'd rather save myself the trouble. All the commits have been cherry-picked in order from the upstream changes, so a code review can compare these commits side by side.
Am 12.07.24 um 13:34 schrieb Herwin Weststrate: Dear Herwin, [...] Thanks a lot for checking this out. > All the commits have been cherry-picked in order from the upstream > changes, so a code review can compare these commits side by side. I'm open to it, but ultimatively it's up to the security team to decide. We can either go for this 100k patch cherry-picked from upstream, or ask for 3.2.5 in stable. Or ignore it, which is in my opinion still on the table (I don't consider BlastRADIUS that bad, but it has a website and a logo so ...) @Security Team: What do you think? Herwin did a spectacular job here already and I can also offer to get it some life testing in a production environment, but in the end we would have to jump into very cold waters. Bernhard
Hi Bernhard, [ I do not think this warrants a DSA, but I see one option, OTOH. How about trying to rebase freeradius to 3.2.5 in the next bookworm point release in august? Then while the issue will not warrant a DSA, we still get the implemented mitigations in a future point release of bookworm. The same obviously could be done as well via a security update, I agree with you assessment that it's not that urgent and so such an update can be batched n the point release and additionally be exposed to the public via the proposed-upates queues. Another story is bullseye, that one is affected as well but a backport there is even harder. For now I have marked it as well no-dsa in the security-tracker, but maybe it should be <ignored> with mentioning that backporting patches is too intrusive? Regards, Salvatore
Hi Bernhard, El 14/07/24 a las 16:15, Salvatore Bonaccorso escribió: set of patches. I've pushed them to: https://salsa.debian.org/debian/freeradius/-/tree/wip/debian/blastradius/bullseye. While they build, I haven't been able to test them (yet). The autopkgtest job fails, but that is related to a bug in Salsa CI and systemd when tmp.mount is masked. Bernhard, are you able to test them? I do not have any experience with FreeRADIUS, so I could test them, but I would take me some time. Just let me know if help is needed here. Cheers,
Hi, I have fixed the autopkgtest on bullseye. I have added a basic test for client with and whitout mitigation. It work. Real testing is needed and a NEWS file for explaining that it is only a bandaid and TLS is better. I plan to backport trixie version to bookworm, and propose a MR if you agree for bookworm. Bastien
Cool, unfortunately I'm off to vacation tomorrow and I'm not sure how much I can do before. I'll be back on August 20th. So, if I understood you correctly, the plan is to use Bastien's backported patches in https://salsa.debian.org/debian/freeradius/-/tree/wip/debian/blastradius/bullseye and update the version in bookworm to the current trixie version, both in a point release? I can test drive the bulleye version on one of our production servers after 20th, and I can certainly ask in the higher education group in Germany who can test either locally available .debs or better use -proposed uploads before the point release. Do we have a date for the next point release already? Bernhard
Le vendredi 9 août 2024, 09:29:44 UTC Bernhard Schmidt a écrit : Ok not a problem and santiago Yes but time here is short, last PU is end of august Fine thansk Bookworm backport could go along ASAP. Risk is low here Last day of august
Hi, I've asked around and found a few who could test drive .debs next week. If you build binaries (or if you have a branch with the proposed changes for bookworm too) I can send them over. okay, so uploading on 23rd at the latest. I'll get to it early CW 34, but you are totally free to give it a go before, no objections. Bernhard
Hello, El 09/08/24 a las 11:57, Bernhard Schmidt escribió: Nice! The binary packages are part of the artifacts from the build jobs, that can be downloaded from: * bullseye: https://salsa.debian.org/debian/freeradius/-/jobs/6100084 * bookworm: https://salsa.debian.org/debian/freeradius/-/jobs/6100049 For convenience, there are deb repositories available, following the instructions found at: * bullseye: https://debian.pages.debian.net/-/freeradius/-/jobs/6100087/artifacts/aptly/index.html * bookworm: https://debian.pages.debian.net/-/freeradius/-/jobs/6100052/artifacts/aptly/index.html ACK! In any case, I would prefer if you could test them in your production severs, so let's see after the 20th. Is that OK for you? Thank you,
I will try to run some tests with these packages. I will have to fit it in between my regular work, so it might take a few days.
I've found one possibly breaking change between the current 3.2.1 and
the proposed 3.2.5: the encoding of binary attributes in JSON. This
might be a fringe issue.
I have used this configuration:
update request {
&Class := "0x313233"
}
rest
This is put in the post-auth section of the default site. The Class
attribute is a binary/octets type attribute, and is added to simplify
reproduction. The rest module has been configured to work with the file
`src/modules/rlm_rest/demo.pl` of the FreeRADIUS repository (but we only
need to look at the request, so just listening with netcat on the
correct port works too). The body type of the rest module is set to
JSON.
With version 3.2.1+dfsg-4+deb12u1 (bookworm stable), the HTTP request
looks like this:
"Class":{"type":"octets","value":["0x313233"]}
Version 3.2.5+dfsg-3~deb12u1 does not add this hex conversion, but
instead uses the textual representation:
"Class":{"type":"octets","value":["123"]}
Non-printable characters are escaped with unicode escaping (I guess
that's the term?), so "0x01" is transmitted as:
"Class":{"type":"octets","value":["\u0001"]}
This change might break things if the REST backend (which is not part of
freeradius itself) expects the hex strings. Our backend was dumb enough
to just strip the first two characters of an octets type attribute
(without checking if they were equal to "0x") and unescape the rest of
the string, and that breaks pretty hard.
The change is done in [1] and I'm not sure how to interpret the bug
report: the second comment say "JSON is not valid", but the JSON string
in the example is perfectly valid.
The change can be reverted by reverting that single line commit linked
in the bug report (I have tested that one). This does keep the behaviour
stable for the Debian bookworm users, but it introduces an
incompatibility with the upstream 3.2.5 version, which can be confusing
when you're reading documentation for the upstream version.
I'm not sure what my advise here would be. Personally, I would love to
see that change reverted simply because it saves me from some work, but
that's not really a valid reason. The change is incompatible with the
current version, but only in very specific setups, so I'm not sure if
anybody else would be affected.
[1] https://github.com/FreeRADIUS/freeradius-server/issues/5285
Le mardi 13 août 2024, 11:54:26 UTC Herwin Weststrate a écrit : I think they said that the type does not correspond to the JSON schema, and I agreed with upstream here. Encoding as hex is an error. JSON5 solve the problem by allowing integer to be encoded as hex but no string. I think it is more a bug fix that need maybe a changelog entry and a warning in the DSA.
The setting `limit_proxy_state` appears to be ignored in the Bullseye
version. The bug can be triggered with the following steps:
* Install the freeradius packages with the instructions listed somewhere
else in this thread.
* Enable the user `bob` in `/etc/freeradius/3.0/users`
* Add an external client to `/etc/freeradius/3.0/clients`. We need an
external client because the `radclient` tool has been updated to
include the `Message-Authenticator` attribute, and we need a request
that does not include that.
* (Re)Start freeradius
* At the external client, install the `freeradius-utils` package from
the current Debian repository (doesn't matter if its Bullseye or
Bookworm, just don't use these new versions from salsa)
Now we can run the first request at the external client:
echo 'User-Name = "bob", User-Password = "hello"' | radclient -x 10.0.0.1 auth testing123
This request should result in the following messages in
`/var/log/freeradius/radius.log`:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
BlastRADIUS check: Received packet without Message-Authenticator.
Setting "require_message_authenticator = false" for client testclient
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
UPGRADE THE CLIENT AS YOUR NETWORK IS VULNERABLE TO THE BLASTRADIUS ATTACK.
Once the client is upgraded, set "require_message_authenticator = true" for this client.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
BlastRADIUS check: Received packet without Proxy-State.
Setting "limit_proxy_state = true" for client testclient
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
The packet does not contain Message-Authenticator, which is a security issue.
UPGRADE THE CLIENT AS YOUR NETWORK MAY BE VULNERABLE TO THE BLASTRADIUS ATTACK.
Once the client is upgraded, set "require_message_authenticator = true" for this client.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
The setting `limit_proxy_state = true` is supposed to forbid requests
from containing a `Proxy-State` attribute. Now if we add this to the
request:
echo 'User-Name = "bob", User-Password = "hello", Proxy-State = 0x313233' | radclient -x 10.0.0.1 auth testing123
This packet gets accepted and you'll see an `Access-Accept` for the
client. The same thing happens when you explicitly configure
`limit_proxy_state = true` for the client, or set this as the global
option.
This settings works as expected in the Bookworm version of the packages.
I've tried it with it with v3.0.x from the freeradius upstream
repository as well, and that too works as expected.
I guess the patches miss an essential part of the code to make it work.
Hello Herwin,
Thanks a lot for testing the proposed packages!
El 15/08/24 a las 17:04, Herwin Weststrate escribió:
Just FTR and completeness, I have been only able to reproduce the issue
when setting `limit_proxy_state = true` for the external client.
In this case, I see this in the radius.log produced by the proposed
package for bullseye:
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: BlastRADIUS check: Received packet without Message-Authenticator.
Error: Setting "require_message_authenticator = false" for client example.org
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: UPGRADE THE CLIENT AS YOUR NETWORK IS VULNERABLE TO THE BLASTRADIUS ATTACK.
Error: Once the client is upgraded, set "require_message_authenticator = true" for this client.
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Otherwise, without setting limit_proxy_state, the packet gets accepted,
and I see a similar error with any of the packages proposed for
bullseye, bookworm, or the packages produced by upstream:
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: BlastRADIUS check: Received packet without Message-Authenticator.
Error: Setting "require_message_authenticator = false" for client example.org
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: UPGRADE THE CLIENT AS YOUR NETWORK IS VULNERABLE TO THE BLASTRADIUS ATTACK.
Error: Once the client is upgraded, set "require_message_authenticator = true" for this client.
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: BlastRADIUS check: Received packet with Proxy-State, but without Message-Authenticator.
Error: This is either a BlastRADIUS attack, OR
Error: the client is a proxy RADIUS server which has not been upgraded.
Error: Setting "limit_proxy_state = false" for client example.org
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error: UPGRADE THE CLIENT AS YOUR NETWORK IS VULNERABLE TO THE BLASTRADIUS ATTACK.
Error: Once the client is upgraded, set "require_message_authenticator = true" for this client.
Error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I am testing the external client with freeradius-utils 3.0.21+dfsg-2.2+deb11u1.
Cheers,
Hi! El 20/08/24 a las 15:14, Santiago Ruano Rincón escribió: https://salsa.debian.org/debian/freeradius/-/commit/e320f4945e88a129d602aad586ac9a927cb344ea Alan, if you ever have some free time, would you be so kind to tell me if that additional patch (for 3.0.21) makes sense? The built packages can be downloaded from: https://salsa.debian.org/debian/freeradius/-/jobs/6156291/artifacts/download, or via the repo as described at: https://debian.pages.debian.net/-/freeradius/-/jobs/6156294/artifacts/aptly/index.html Herwin, if possible, could you please give it a try? I think the behaviour matches the upstream's bookworm, but I would be great to have an extra pair of eyes :-) Cheers,
The patch looks good to me, thanks.
It looks like this has fixed it. This was the last issue I found, so both versions got my stamp of approval. I'll be on holiday tomorrow until the end of the month, so if a new version will be released, I won't have a chance to look at it.
El 22/08/24 a las 17:14, Herwin Weststrate escribió: Herwin, thank you for testing! Much appreciated :-) El 21/08/24 a las 13:58, Alan DeKok escribió: Thank you, Alan! Bernhard, Bastien (in CC) is willing to handle the PUs. Please, let us know if it is still OK for you. In any case, we would like to avoid duplicated work. Cheers,
Sure, go ahead. @Bastien feel free to push to the normal branches in Salsa as well. No objections in any way. FTR, I've tested the binaries on our radius setup today and they worked as expected. Bernhard
Unfortunately I'm still lacking time, but today I had two unexpected consequences. A clients.conf entry spanning large subnets was upgraded automatically from require_message_authenticator auto -> yes due to the first package being received. Consequently messages from another Radius client within the same clients.conf entry was dropped silently. So far as expected, but I would have assumed FreeRADIUS to log an error when a request without Message-Authenticator attribute comes in and it is (auto-)configured to expect one. But I did not see anything. Is this correct? Another thing to watch out, although I would not want it to be in the official changelog/news, Checkpoint Firewalls are known to be broken by FreeRADIUS returning a Message-Authenticator attribute, see https://support.checkpoint.com/results/sk/sk42184 . Apparently there is an internal workaround available, but only to paying users. I could not find a quick way to disable FreeRADIUS always _sending_ the Message-Authenticator header. All of that only quickly tested on the Bullseye package, I had no time yet to dig deeper. Bernhard
Yes, clients which aren't /32 should likely be left as `auto`, which really means `no`. It logs it in debug mode, but not in normal mode. I suppose that could be changed, but it would have to be rate-limited. There's a documented work-around. https://support.checkpoint.com/results/sk/sk182516 I can only put this down to complete incompetence. The Message-Authenticator attribute has been defined for about 25 years. There is *zero* reason to drop packets which contain Message-Authenticator. The whole idea of dropping packets which contain unknown attributes is stupid. After running into this issue, I've updated the "fixing RADIUS security" RFC: https://datatracker.ietf.org/doc/html/draft-ietf-radext-deprecating-radius-03#name-unknown-attributes It can't be disabled. We're not going to make 1,000,000 networks insecure simply because one vendor hasn't updated their RADIUS implementation in 25 years. There is a work-around for the checkpoint issue, so additional configuration changes for FreeRADIUS aren't a good idea. Alan DeKok.
Hello, For information, among radius clients that we have in Debian, at least openvpn-auth-radius and l2tpns need to be fixed into sending & checking the Message-Authenticator option. Samuel