Upstream recently added support for the following Pine64 Quartz64 devices:
- Quartz64 Model A
- Quartz64 Model B
- SoQuartz on Model A board
- SoQuartz on Blade board
- SoQuartz on CM4 IO carrier board

Link: https://source.denx.de/u-boot/u-boot/-/tree/master/board/pine64/quartz64_rk3566

Hereby the request to package them for Debian.

TIA,
  Diederik

Would you or someone else be willing to be listed as a tester for these
boards?

https://wiki.debian.org/U-boot/

Presuming they are similar to the other rockchip arm64 targets, could
you provide a basic patch and test one or more of the above platforms to
work? For example, the rock64-rk3328:

https://salsa.debian.org/debian/u-boot/-/blob/debian/latest/debian/targets.mk#L109

Although, you will likely first need to enable in arm-trusted-firmware,
as that is usually a build-dependency for u-boot on rockchip platforms.

live well,
  vagrant

Would you or someone else be willing to be listed as a tester for these
boards?

https://wiki.debian.org/U-boot/

Presuming they are similar to the other rockchip arm64 targets, could
you provide a basic patch and test one or more of the above platforms to
work? For example, the rock64-rk3328:

https://salsa.debian.org/debian/u-boot/-/blob/debian/latest/debian/targets.mk#L109

Although, you will likely first need to enable in arm-trusted-firmware,
as that is usually a build-dependency for u-boot on rockchip platforms.

live well,
  vagrant

Hi Vagrant,

You can add me as tester for Quartz64 Model A + B, but I don't have a SoQuartz
(or a base/carrier board).

I can/will give it a try, but the u-boot stuff is mostly a mystery to me.
https://github.com/Kwiboo/u-boot-build/ is what I have been using so far ...
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/16952
and https://review.trustedfirmware.org/c/ci/tf-a-ci-scripts/+/20480 (although
it does seem to have gotten some momentum recently)

In some build instructions for u-boot that I've seen there was a reference to
an `bl31.elf` file from https://github.com/rockchip-linux/rkbin but I guess
that's what TF-A would provide (too), but then built from source?

I also recall seeing references to a `rk3568_ddr_1056MHz_v1.18.bin` file from
https://github.com/rockchip-linux/rkbin/tree/master/bin/rk35. AFAIK that's
only available as a BLOB? Is that file needed? And is it a problem if that's
only available as a BLOB?
Rockchip did add a LICENSE recently (in case that helps/is relevant):
https://github.com/rockchip-linux/rkbin/blob/master/LICENSE

Cheers,
  Diederik

Ok, if we ever get there... but I would not want to enable new boards
that nobody has ever offered to test.

This is going to require quite a bit of work:

* package "rkbin" in non-free-firmware, if the license permits

* get appropriate rk35xx builds for TF-A (a.k.a. arm-trusted-firmware)
  merged upstream, and then uploaded to the debian package (can possibly
  cheat and instead only use the bl31.elf from rkbin).

* figure out the complicated dance between packages in "contrib"
  depending on packages in "non-free-firmware" and not polluting
  packages built from the same source in "main"

Yeah, there does at least appear to be some recent activity, at least...

Yes.

If upstream u-boot requires that to work it is... more complicated.

Looks like it does probably require it, based on the trustedfirmware
thread you linked to above.

That license looks like it *maybe* would work for the
"non-free-firmware" part of the repo...

In which case, it might be possible to use those blobs for both
arm-trusted-firmware/TF-A and the ddr training blob.


Someone had briefly explored doing something similar for imx8*
platforms, and it was a maze of complications... though the imx8* blobs
turned out to not even be suitible for non-free-firmware, in the end, so
it stalled out.

Because of all these complications, I have so far tried to stick to
platforms which are strictly suitable for debian "main", although
technically the requirements are a bit looser than they used to
be... but I have little experience with or enthusiasm for the
complexities such a setup would bring for the u-boot and
arm-trusted-firmware packaging in Debian.


live well,
  vagrant

I can probably get one or more people to test, but I doubt I can get them
listed as 'official testers'.

AFAIK, which isn't much in this case, it should be the same/similar to rk3328/
rk3399? They also use a file similar to `rk3568_ddr_1056MHz_v1.18.bin`, but
then ofc tailored for their platform.
I'll ask the upstream maintainer/reviewer; this bug is probably not the place
to document my learning experiences ;)

Ok, thanks.

As mentioned above, AFAIK it's no different from rk3328/rk3399, but I'll ask.

Let's for now assume that what I said was inaccurate.
Trying to make it work, assuming it's similar to rk3328, is going to be a
challenge enough. If it'll require some 'hustling' between main/contrib/non-
free/non-free-firmware, I'll probably bail out.

Cheers,
  Diederik

If the rk3568*.bin is not actually required to build an upstream
arm-trusted-firmware, then it is not different!

If the rk3568*.bin is required to build an upstream
arm-trusted-firmware, then it is different!

The current arm-trusted-firmware and u-boot packages for rk3328 and
rk3399 do not require those or other blobs.

live well,
  vagrant

I asked and it *is* different ... and thus (a lot) more complicated.

I thought it would be the same as rk3328/rk3399 as that also uses a DRAM init/
training. While that itself is true, "rk3328 and rk3399 can use U-Boot's open
source dram init", while rk356x's dram init is not open source, but provided
under the non-free (AFAICT, but IANAL) license provided here:
https://github.com/rockchip-linux/rkbin/blob/master/LICENSE

So it looks like you were right and `rkbin` needs to be packaged separately
and then combined with the rest, which I think is all Free Software.
I guess `rkbin` can be put in non-free-firmware, but I don't know in which
archive area an u-boot package which uses/needs that should be placed.

You too :-)

Hi,

This is the case for some other devices (e.g. rk3588) currently where there is support in U-Boot mainline but there is no arm-trusted-firmware support _or_ DDR bringup in U-Boot as yet.

There are two options I can see:

1) Workaround this by building the "open-source" part of U-Boot for these devices and letting users build their own final U-Boot binary with closed source parts. The licence now allows redistribution:
https://github.com/rockchip-linux/rkbin/blob/master/LICENSE
2) Wait for arm-trusted-firmware and u-boot DDR bringup code to be open-sourced and merged into mainline. This could take some months to several years.


If anyone is interested in option 1, I could be convinced to package rkbin in non-free-firmware/non-free possibly along with additional scripts to make things build...
It'd be great to get Debian running on some of this more recent Rockchip hardware while we wait for Rockchip to sort out option 2. I've been told it's in progress ;-)


Thanks!

Chris

rather interesting for rk3588 support.
I'm less 'in the loop' wrt rk3588 on u-boot and TF-A although I did post this:
https://forum.pine64.org/showthread.php?tid=14432&pid=117094#pid117094

I do expect for TF-A support to land upstream for rk3588 (eventually), but do
you know (or suspect) that DRAM init will be open sourced too?

Note that I don't _fully_ understand these things, so if what I'm saying
doesn't make any sense, feel free to point that out :-)

I'm currently working on that. Although it's initially for Quartz64 Model A+B
and PineTab2 (all rk3566) devices, I don't see why it wouldn't be usable for
rk3588 devices too as (long as) they're all arm64.

AFAICT (now), I can just make 1 image as the only differentiator would be the
specific u-boot binary that needs to be put in place and I can just make an
instruction to `dd` the right u-boot binary onto sector 64 of the image file.

Thus https://github.com/Kwiboo/u-boot-build/#flashing with `/dev/mmcblk0`
replaced by my image file.

https://salsa.debian.org/diederik/linux/-/tree/pine64/master-next is the
kernel I (currently) use, which is based (and periodically rebased up) on the
Debian kernel (with the intend to 'upstream' it to Debian's kernel when that's
appropriate).

Cheers,
  Diederik

Hi Diederik,

Sorry, our mails collided and I didn't see your previous reply.

There are a few open pull requests from Rockchip to add support but it is likely
to take a while to be merged.

I am not entirely sure of Rockchip's plans here.

I have added Kever to the thread who can hopefully suggest when DRAM init code
and TF-A support for various Rockchip processors will be submitted to mainline?

So far so good ;-). The only clarification I would like to make is the DDR init code
is standalone, part of the TPL and is not anything to do with TF-A. It really is the
first-stage boot code that runs after the maskrom.

We could use Rockchip's closed implementation of that OR U-Boot's implementation, it's
only real job is to train the DDR and handoff to U-Boot SPL.
1) build and distribute U-Boot for these "new" targets without the TPL/TF-A
2) package rkbin for Debian in non-free-firmware (I will happily do that)
3) have some scripts to collate the various bits into proper images (that probably
should go inside the image-building tool rather than in a Debian package like
flash-kernel, since it will be hopefully obsolete one day :-))

Great! I am interested in bringing up more of these boards in Debian too.
I have some custom Debos recipes to build some images for these boards, but the
recipes really aren't yet ready to go anywhere until we get kernel and bootloader
sorted out.


Thanks!

Chris

Hi Christopher,

Ha! I thought about sending you a private mail about that, but chose not to as
they weren't conflicting.

Thanks! Then it looks like I'm closer to understanding then I initially
thought. It gives a 405 since yesterday, but I have been studying
https://opensource.rock-chips.com/wiki_Boot_option

AFAICT, TF-A for *rk356x* is really close to being mergeable? It appears to me
that all that's needed is to "dot the i's" and the right people pushing the
right 'buttons' (in the right sequence).
AFAIC we can wait for that to happen.

The situation is different and therefor long(er) term when it comes to TF-A for
rk3588 based devices. I'd like to see support for that in Debian 'eventually'
too, but strictly speaking that's not the goal of this bug report.

The tricky thing is TPL/DRAM training.

Thanks :-)
AFAIUI the tricky part is about combining u-boot from 'main' (which Vagrant
likes to keep in 'main' and I'm inclined to agree) with rkbin from non-free-
firmware and *IF* that's allowed in 'main'. If not, what then?
Create an u-boot-nff package, with (exact?) the same contents as u-boot, but
just in a different archive area? So that it can then be combined to create an
u-boot-rockchip-nff package (in n-f-f)?

My goal is to have all the parts *in* Debian as packages, so that I don't need
to run a script to get/build u-boot for rk356x devices, but can just do
`apt-get install <u-boot-for-rk356x>`

So the image-building is just used to build/'assemble' the image ;-)

Just like the goal of that branch is to 'disappear' as all the needed parts
have (then) become part of the official Debian kernel.
We're not there yet, but that is my goal (for the Trixie release).

Cheers,
  Diederik

Hi Diederik,

To stay on-topic here, I also have a Pine64 Quartz64 model A - I'd be happy to be added as a board maintainer in Debian.

I didn't yet manage to build U-Boot mainline for it, see https://lists.denx.de/pipermail/u-boot/2023-August/526625.html
I still owe a reply on that topic to Jonas' suggestions, I will hope to get to that next week.

No problem, on-list is probably best anyway so it is archived ;-)

Yeah, I also got stuck with the vendor documentation and eventually just *had a go* and eventually manged to get things working ;-).
with vendor TPL & leave out TF-A entirely ;-). Maybe that could even be possible with this board,
to just run without TF-A ?

If you want to do that, see how we do it for rk3588 here: https://gitlab.collabora.com/hardware-enablement/rockchip-3588/u-boot/-/blob/rk3588-rock5b/.gitlab-ci.yml
I don't know if it will work for Quartz64, but please try it and see. My fingers are crossed.

Right, see my comment above. Maybe we can just ignore TF-A on these devices *for now*
until upstream is merged?

Right, that is common between all of these "modern" devices, so I am really trying to
not derail this into a rk3588 support thread, but that should be in a separate bug report.
In theory, if we skip TF-A, the only thing we need is TPL from rkbin for both boards.

Let me rephrase myself. My suggestion is to ship in u-boot-rockchip U-Boot for these devices
*without* any closed bits and ship rkbin in non-free-firmware in non-free-firmware.

We need to simply combine those into an image file to flash to the device, that could be done
in flash-kernel (or as my original suggestion in a script in the image building process but I am
not fussy).

So that script would "just" combine rkbin and the "open" parts of u-boot-rockchip into idbloader.img and u-boot.itb.
And once the DDR init and AT-F bits are merged into mainline, we can remove those bits from flash-kernel
and u-boot-rockchip package can be extended to use the mainline code.

@Vagrant, does that sound like a solution which you'd be interested in? I could
try to carve out some time to post a patch for u-boot, flash-kernel and package rkbin if so ;-).

Fingers crossed!

Thanks,

Chris

Hi Diederik,

Sorry, I was wrong. We set BL31 (which is TF-A from rkbin) and ROCKCHIP_TPL (which is TPL from rkbin)

e.g:
    - export BL31=../rkbin/bin/rk35/rk3588_bl31_v1.27.elf
    - export ROCKCHIP_TPL=../rkbin/bin/rk35/rk3588_ddr_lp4_2112MHz_lp5_2736MHz_v1.08.bin

That was misleading. Apologies.

That was wrong. I believe we can use TF-A from rkbin and combine things into an image later.


That was wrong. I believe we can use TF-A from rkbin and combine things into an image later.


Thanks!

Chris

That is definitely one approach, although we would need a
u-boot-rockchip-something variant that goes to "contrib" instead of
"main". Packages in "main" need to be functional without anything
outside of "main", and obviously missing the rkbin bits would be of
limited use. :)

There is at least one existing u-boot variant that need to move to
contrib for this very reason...

Another option would be to ship a u-boot-source.deb, which contains the
upstream sources, much like the linux-source package. Then you can maybe
ship u-boot-contrib and/or u-boot-non-free-firmware that builds the
source and can depend on rkbin.

I was so glad rk3328 and rk3399 and earlier rk3288 did not have these
sorts of issues, and was sad to see new ones for rk35xx ...

josch@debian.org and I (mostly josch) hashed out some similar ideas for
an imx8* platform with similar issues (although the licensing ended up
being even worse than for rkbin such that it could not even go to
"non-free-firmware"):

https://salsa.debian.org/debian/u-boot/-/merge_requests/29

live well,
  vagrant

I'm not really familiar with Gerrit, but it appears to me that someone
needs to review and if it's OK, give a '+1' for Code-Review. And for the
CI to run, someone needs to '+1' for Allow-CI. And possibly similar
things for the 'Hashtags'/'Other labels'. That's what I meant by
"pushing the right 'buttons' (in the right sequence)".
for rk3588 is unknown to me and could be totally different.

AFAICT, there are 2 'MRs' for TF-A for rk356x:
1) Addition to the (CI) build config
2) The actual patch set for TF-A for rk356x

ad 1) https://review.trustedfirmware.org/c/ci/tf-a-ci-scripts/+/20480
AFAICT the (minor) review comments have been addressed in patchset 2.
It may need a fix as identified by '2'.

ad 2) https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/16952
Depends on 1 AFAICT. It's missing a SCMI part, but the plan is to add
that in a subsequent patch.
What I have identified as issues are the following:
- needs to be rebased on current `master`
- commit title: `s/^:feat:/feat:/` (IOW drop the preceding ':')
- includes need to be in a specific order; see
https://trustedfirmware-a.readthedocs.io/en/latest/process/coding-style.html#include-statement-ordering
- `plat/rockchip/rk3568/drivers/pmu/pmu.c`,
  `plat/rockchip/rk3568/drivers/soc/soc.c` and
  `plat/rockchip/rk3568/plat_sip_calls.c` need to be added to
  `tf-cov-make` (see '1')
- Extend commit message wrt changes in `lib/cpus/aarch64/cortex_a55.S`
- In `plat/rockchip/rk3568/drivers/pmu/pmu.c`, "Use braces { } even for single
  line statements"

And then trigger the CI again and address any issues that come up.

That does not sound like an enormous amount of work to me.
If then the right people give their '+1' on the various categories, then
it could be merged?

Cheers,
  Diederik

The latter one has just been merged \o/
Thanks Kever and shengfei!

The former one is still 'Active' although I thought they were supposed to be
merged together.
And apparently 'Gerrit' is a bit confused (by TF-A's workflow?)

https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 for rk3588
now has a Merge Conflict (due to merging the rk3568 stuff?), but that has seen
activity in the last month too, so hopefully that'll progress (soon) too.

Cheers,
  Diederik

Support for rk3588 has been merged upstream ... yesterday \o/