* [PATCH] Add image configuration option
@ 2022-09-15 7:35 José Pekkarinen
2022-09-15 8:21 ` Alyssa Ross
0 siblings, 1 reply; 13+ messages in thread
From: José Pekkarinen @ 2022-09-15 7:35 UTC (permalink / raw)
To: devel; +Cc: José Pekkarinen
The following patch proposes to host nix configuration
files under nix folder that offers default configuration
for an image, defaulting to a release image, which would
be plain spectrum. A hardened default configuration will
be proposed in the near future. In case of configuration
collision between the default configuration and config.nix,
the latter will be taken into account.
Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
---
nix/eval-config.nix | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/nix/eval-config.nix b/nix/eval-config.nix
index 467f877..09a7ada 100644
--- a/nix/eval-config.nix
+++ b/nix/eval-config.nix
@@ -10,4 +10,5 @@
({ pkgs ? import <nixpkgs> {} }: {
inherit pkgs;
-}) config
+}) (if config ? image && builtins.pathExists ./${config.image}.nix then
+ import ./${config.image}.nix else {}) // config
--
2.25.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 7:35 [PATCH] Add image configuration option José Pekkarinen
@ 2022-09-15 8:21 ` Alyssa Ross
2022-09-15 10:42 ` José Pekkarinen
0 siblings, 1 reply; 13+ messages in thread
From: Alyssa Ross @ 2022-09-15 8:21 UTC (permalink / raw)
To: José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 2470 bytes --]
José Pekkarinen <jose.pekkarinen@unikie.com> writes:
> The following patch proposes to host nix configuration
> files under nix folder that offers default configuration
> for an image, defaulting to a release image, which would
> be plain spectrum. A hardened default configuration will
> be proposed in the near future. In case of configuration
> collision between the default configuration and config.nix,
> the latter will be taken into account.
>
> Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
> ---
> nix/eval-config.nix | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
Hi José, thanks for the patch!
It looks like the correct way to implement such a feature, but I'm not
sure about the feature itself.
Currently we only have a single configuration option, pkgs. So it
doesn't make sense to be able to split build configuration across two or
more files, because only one of them would be able to set the one
configuration option that exists so far.
We could end up with more configuration options, of course, but I'd
really like to avoid the situation where a Spectrum build configuration
is so complicated it needs to be expressed across multiple files in this
way. Sometimes configuration is unavoidable, like how we have to give
people a way to use a vendor kernel if required, because we can't
possibly bundle every vendor kernel we might want to use into the same
image, but using build configuration should really be a last resort.
I'd expect very few Spectrum users overall to be building their own
images, so the most important thing is for the default configuration to
be as good as possible. Hardening falls under that — if we can do
something to harden the Spectrum system, we should probably be doing it
by default! Or if it's something that doesn't make sense to do by
default, can we make it configurable at runtime so that users don't have
to build their own images if they want to use it? (I'm hoping the
proposed developer mode could work this way, for example. I haven't
thought about it enough to know if it's practical, but Chrome OS can do
it.)
If we ever do end up with lots of configuration options to the point
where they're getting difficult to manage, we can re-evaluate something
like this (or at that point it might just be worth it to give in and
reuse the NixOS module system), but I don't think we're at that point
yet.
What do you think?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 8:21 ` Alyssa Ross
@ 2022-09-15 10:42 ` José Pekkarinen
2022-09-15 11:31 ` Alyssa Ross
0 siblings, 1 reply; 13+ messages in thread
From: José Pekkarinen @ 2022-09-15 10:42 UTC (permalink / raw)
To: Alyssa Ross; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 4919 bytes --]
On Thu, Sep 15, 2022 at 11:21 AM Alyssa Ross <hi@alyssa.is> wrote:
> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>
> > The following patch proposes to host nix configuration
> > files under nix folder that offers default configuration
> > for an image, defaulting to a release image, which would
> > be plain spectrum. A hardened default configuration will
> > be proposed in the near future. In case of configuration
> > collision between the default configuration and config.nix,
> > the latter will be taken into account.
> >
> > Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
> > ---
> > nix/eval-config.nix | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
>
> Hi José, thanks for the patch!
>
> It looks like the correct way to implement such a feature, but I'm not
> sure about the feature itself.
>
> Currently we only have a single configuration option, pkgs. So it
> doesn't make sense to be able to split build configuration across two or
> more files, because only one of them would be able to set the one
> configuration option that exists so far.
>
> We could end up with more configuration options, of course, but I'd
> really like to avoid the situation where a Spectrum build configuration
> is so complicated it needs to be expressed across multiple files in this
> way. Sometimes configuration is unavoidable, like how we have to give
> people a way to use a vendor kernel if required, because we can't
> possibly bundle every vendor kernel we might want to use into the same
> image, but using build configuration should really be a last resort.
>
> I'd expect very few Spectrum users overall to be building their own
> images, so the most important thing is for the default configuration to
> be as good as possible. Hardening falls under that — if we can do
> something to harden the Spectrum system, we should probably be doing it
> by default! Or if it's something that doesn't make sense to do by
> default, can we make it configurable at runtime so that users don't have
> to build their own images if they want to use it? (I'm hoping the
> proposed developer mode could work this way, for example. I haven't
> thought about it enough to know if it's practical, but Chrome OS can do
> it.)
>
> If we ever do end up with lots of configuration options to the point
> where they're getting difficult to manage, we can re-evaluate something
> like this (or at that point it might just be worth it to give in and
> reuse the NixOS module system), but I don't think we're at that point
> yet.
>
> What do you think?
>
Hi,
In my humble opinion, considering that we are working at
an operative system level, the idea of having a default configuration,
and a debug configuration, preferably that we can activate at runtime,
was outnumbered, but not today, twenty years ago. Only thinking in
our current workflows, we can easily spot variables enough to show
that 2 configurations are not sufficient, you already gave a good
example with the vendor kernels, I can give some more, like building
for a vm, or a host, of arch x86_64 or arm, natively built, or cross
compiled, hardened or not, with debugging tools and symbols or
without them, with extra vms to run application X, without wayland
and graphics applications because our target machine doesn't have
a display. Several of these cases can be combined, and multiple
of them requires changes at build time not giving the option to
enable them at runtime, so eventually not only you'll have a big
set of configuration files, but you'll also want to combine them in
a smart way so that the complexity is bearable and your user
base doesn't get upset because it is very hard to handle.
Now, I have to fully disagree that we are not in the
point were we need to re-evaluate things, we are, and you were
before we joined you, you only need to take a look at the contribution
level of your project in time, since the effect of not implementing
flexibility to developers to make their own Spectrum OS for their
needs, is that you will end up with a big base of user that forks
Spectrum with a feature that is divergent enough that proposing
it back to the upstream is unlikely, or even more dangerous, that
the developer in question doesn't even know how to implement
the feature they are required, and they don't even try to hack
the system, loosing the contribution since the beginning. Needless
to mention, you can easily find examples of these 2 scenarios
inside Unikie, no need to look further. Examples in how this
flexibility can be implemented in multiple ways, and it is a
requirement to guarantee a bigger volume of contributions can
be found in communities like buildroot, yocto, or even Nix.
I hope these give you good hints, and if it doesn't, that
we friendly agree to disagree.
José.
[-- Attachment #2: Type: text/html, Size: 6411 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 10:42 ` José Pekkarinen
@ 2022-09-15 11:31 ` Alyssa Ross
2022-09-15 12:31 ` José Pekkarinen
0 siblings, 1 reply; 13+ messages in thread
From: Alyssa Ross @ 2022-09-15 11:31 UTC (permalink / raw)
To: José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 6801 bytes --]
José Pekkarinen <jose.pekkarinen@unikie.com> writes:
> On Thu, Sep 15, 2022 at 11:21 AM Alyssa Ross <hi@alyssa.is> wrote:
>
>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>
>> > The following patch proposes to host nix configuration
>> > files under nix folder that offers default configuration
>> > for an image, defaulting to a release image, which would
>> > be plain spectrum. A hardened default configuration will
>> > be proposed in the near future. In case of configuration
>> > collision between the default configuration and config.nix,
>> > the latter will be taken into account.
>> >
>> > Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
>> > ---
>> > nix/eval-config.nix | 3 ++-
>> > 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> Hi José, thanks for the patch!
>>
>> It looks like the correct way to implement such a feature, but I'm not
>> sure about the feature itself.
>>
>> Currently we only have a single configuration option, pkgs. So it
>> doesn't make sense to be able to split build configuration across two or
>> more files, because only one of them would be able to set the one
>> configuration option that exists so far.
>>
>> We could end up with more configuration options, of course, but I'd
>> really like to avoid the situation where a Spectrum build configuration
>> is so complicated it needs to be expressed across multiple files in this
>> way. Sometimes configuration is unavoidable, like how we have to give
>> people a way to use a vendor kernel if required, because we can't
>> possibly bundle every vendor kernel we might want to use into the same
>> image, but using build configuration should really be a last resort.
>>
>> I'd expect very few Spectrum users overall to be building their own
>> images, so the most important thing is for the default configuration to
>> be as good as possible. Hardening falls under that — if we can do
>> something to harden the Spectrum system, we should probably be doing it
>> by default! Or if it's something that doesn't make sense to do by
>> default, can we make it configurable at runtime so that users don't have
>> to build their own images if they want to use it? (I'm hoping the
>> proposed developer mode could work this way, for example. I haven't
>> thought about it enough to know if it's practical, but Chrome OS can do
>> it.)
>>
>> If we ever do end up with lots of configuration options to the point
>> where they're getting difficult to manage, we can re-evaluate something
>> like this (or at that point it might just be worth it to give in and
>> reuse the NixOS module system), but I don't think we're at that point
>> yet.
>>
>> What do you think?
>>
>
> Hi,
>
> In my humble opinion, considering that we are working at
> an operative system level, the idea of having a default configuration,
> and a debug configuration, preferably that we can activate at runtime,
> was outnumbered, but not today, twenty years ago. Only thinking in
> our current workflows, we can easily spot variables enough to show
> that 2 configurations are not sufficient, you already gave a good
> example with the vendor kernels, I can give some more, like building
> for a vm, or a host, of arch x86_64 or arm, natively built, or cross
> compiled, hardened or not, with debugging tools and symbols or
> without them, with extra vms to run application X, without wayland
> and graphics applications because our target machine doesn't have
> a display. Several of these cases can be combined, and multiple
> of them requires changes at build time not giving the option to
> enable them at runtime, so eventually not only you'll have a big
> set of configuration files, but you'll also want to combine them in
> a smart way so that the complexity is bearable and your user
> base doesn't get upset because it is very hard to handle.
Okay, thanks for the explanation. I think we can group some of these
together:
• Stuff that's already Nixpkgs configuration options or can be
expressed through an overlay. Whether to cross compile, what
architecture to build for, whether to use a vendor kernel, etc. This
can already be handled through the existing configuration mechanism.
• VM customisation, including extra VMs, disabling Wayland, etc. In my
mind there are still some open questions around how this should be
implemented exactly, but this is definitely something that needs to
be more configurable.
• Whether to install extra stuff on the host system. This covers
things like debugging symbols and tools.
Does that sound right to you? Are there more that you can think of?
I'd like to understand the requirements here better, to help think about
what sort of configuration mechanisms might be required.
> Now, I have to fully disagree that we are not in the
> point were we need to re-evaluate things, we are, and you were
> before we joined you, you only need to take a look at the contribution
> level of your project in time, since the effect of not implementing
> flexibility to developers to make their own Spectrum OS for their
> needs, is that you will end up with a big base of user that forks
> Spectrum with a feature that is divergent enough that proposing
> it back to the upstream is unlikely, or even more dangerous, that
> the developer in question doesn't even know how to implement
> the feature they are required, and they don't even try to hack
> the system, loosing the contribution since the beginning. Needless
> to mention, you can easily find examples of these 2 scenarios
> inside Unikie, no need to look further. Examples in how this
> flexibility can be implemented in multiple ways, and it is a
> requirement to guarantee a bigger volume of contributions can
> be found in communities like buildroot, yocto, or even Nix.
You're right — it's generally a bad thing if people have to patch
Spectrum to make it fit their needs. I want to avoid that. But most of
the things we've talked about so far don't feel to me like they're going
to lead to massive configuration files. The exception is all the stuff
that's either Nixpkgs configuration or overlays, but the mechanism you
proposed here wouldn't help with that, because only a single
configuration file would be able to change anything in "pkgs", due to
the configuration files being merged using the non-recursive //
operator. That's exactly why it's important to understand what the
needs are before we consider specific configuration mechanisms. It's
difficult to figure out if an idea will actually make things easier
without seeing an example of the problem, and what difference to it the
proposed solution would make.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 11:31 ` Alyssa Ross
@ 2022-09-15 12:31 ` José Pekkarinen
2022-09-15 13:22 ` Alyssa Ross
0 siblings, 1 reply; 13+ messages in thread
From: José Pekkarinen @ 2022-09-15 12:31 UTC (permalink / raw)
To: Alyssa Ross; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 2959 bytes --]
On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
> [...]
> Okay, thanks for the explanation. I think we can group some of these
> together:
>
> • Stuff that's already Nixpkgs configuration options or can be
> expressed through an overlay. Whether to cross compile, what
> architecture to build for, whether to use a vendor kernel, etc. This
> can already be handled through the existing configuration mechanism.
>
> • VM customisation, including extra VMs, disabling Wayland, etc. In my
> mind there are still some open questions around how this should be
> implemented exactly, but this is definitely something that needs to
> be more configurable.
>
> • Whether to install extra stuff on the host system. This covers
> things like debugging symbols and tools.
>
> Does that sound right to you? Are there more that you can think of?
> I'd like to understand the requirements here better, to help think about
> what sort of configuration mechanisms might be required.
>
This is a good summary, currently I can think of a coupel of more,
which is a mechanism to provide upstream project configuration artifacts
when nix allow to bypass their automated configs, for example, a kernel
config when using manual config kernel nix package, a bit more unclear
is how to provide upstream configs when nix is not flexible enough. A
mechanism to generate a full image from the nix generated artifacts putting
together kernel, initrd, rootfs and ext partition so that the full image can
be flashed in a sdcard of choice and use it. This would require to be
configurable so that you can modify the partition table to suit vendor
needs.
[...]
> You're right — it's generally a bad thing if people have to patch
> Spectrum to make it fit their needs. I want to avoid that. But most of
> the things we've talked about so far don't feel to me like they're going
> to lead to massive configuration files. The exception is all the stuff
> that's either Nixpkgs configuration or overlays, but the mechanism you
> proposed here wouldn't help with that, because only a single
> configuration file would be able to change anything in "pkgs", due to
> the configuration files being merged using the non-recursive //
> operator. That's exactly why it's important to understand what the
> needs are before we consider specific configuration mechanisms. It's
> difficult to figure out if an idea will actually make things easier
> without seeing an example of the problem, and what difference to it the
> proposed solution would make.
>
Well, the exact point of that is to give you a small next step to move
towards a more flexible configuration system, without loosing control
to revert back if the situation doesn't look better. It is unlikely a more
complex contribution would be acceptable for you if even a little one
like this wouldn't make it.
José.
[-- Attachment #2: Type: text/html, Size: 4110 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 12:31 ` José Pekkarinen
@ 2022-09-15 13:22 ` Alyssa Ross
2022-09-15 13:48 ` Ville Ilvonen
2022-09-16 4:59 ` [PATCH] Add image configuration option José Pekkarinen
0 siblings, 2 replies; 13+ messages in thread
From: Alyssa Ross @ 2022-09-15 13:22 UTC (permalink / raw)
To: José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 6015 bytes --]
José Pekkarinen <jose.pekkarinen@unikie.com> writes:
> On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>
>> [...]
>> Okay, thanks for the explanation. I think we can group some of these
>> together:
>>
>> • Stuff that's already Nixpkgs configuration options or can be
>> expressed through an overlay. Whether to cross compile, what
>> architecture to build for, whether to use a vendor kernel, etc. This
>> can already be handled through the existing configuration mechanism.
>>
>> • VM customisation, including extra VMs, disabling Wayland, etc. In my
>> mind there are still some open questions around how this should be
>> implemented exactly, but this is definitely something that needs to
>> be more configurable.
>>
>> • Whether to install extra stuff on the host system. This covers
>> things like debugging symbols and tools.
>>
>> Does that sound right to you? Are there more that you can think of?
>> I'd like to understand the requirements here better, to help think about
>> what sort of configuration mechanisms might be required.
>
> This is a good summary, currently I can think of a coupel of more,
> which is a mechanism to provide upstream project configuration artifacts
> when nix allow to bypass their automated configs, for example, a kernel
> config when using manual config kernel nix package, a bit more unclear
> is how to provide upstream configs when nix is not flexible enough.
You mean you'd like to manually provide a Kconfig file, rathen than
using Nixpkgs' (not very good) structured config mechanism, right?
That should be possible with an overlay, but maybe some documentation
with an example would be a good idea?
> A mechanism to generate a full image from the nix generated artifacts
> putting together kernel, initrd, rootfs and ext partition so that the
> full image can be flashed in a sdcard of choice and use it. This would
> require to be configurable so that you can modify the partition table
> to suit vendor needs.
We've had some discussion about that already on the list and on IRC. My
current view is that early boot firmware (U-Boot etc.) doesn't really
have anything to do with Spectrum. They both run at different times,
and they communicate over a standard interface (EBBR [1]), so the
specifics of the firmware aren't really in scope for Spectrum itself and
belong elsewhere. It doesn't make sense for Spectrum to be installing
U-Boot any more than it makes sense for U-Boot to be installing
Spectrum, or for Linux to be installing U-Boot — they are two separate
components. (This isn't an approach unique to Spectrum — Fedora is
doing something similar.)
It can make sense to make an image that is a combination of U-Boot and
Spectrum, but that process should be part of an integration between the
two that exists one layer up, rather than part of either project. For
example, you could do something like this:
let
spectrum = import <spectrum/img/live> {
# config could either be loaded using the standard mechanism
# or inlined here.
};
# It would also be possible to import individual components of
# Spectrum and assemble them manually if even greater
# flexibility was required, but I doubt that would be common.
inherit (spectrum) pkgs;
# I don't think a pkgs attribute currently exists on the
# spectrum-live.img derivation, but it might make sense to add
# for this sort of use case.
uboot = pkgs.ubootRockPro64;
in
pkgs.runCommand "uboot+spectrum.img" {} ''
# Use sfdisk (or maybe there's some better tool)
# to create a partition table, and copy the U-boot image
# and the Spectrum images into place.
#
# Spectrum is designed to accomodate this by not expecting
# any of its partitions to be at any particular location
# on disk.
''
Maybe this is another case where documentation and a worked example
would help?
[1]: https://arm-software.github.io/ebbr/index.html
> [...]
>> You're right — it's generally a bad thing if people have to patch
>> Spectrum to make it fit their needs. I want to avoid that. But most of
>> the things we've talked about so far don't feel to me like they're going
>> to lead to massive configuration files. The exception is all the stuff
>> that's either Nixpkgs configuration or overlays, but the mechanism you
>> proposed here wouldn't help with that, because only a single
>> configuration file would be able to change anything in "pkgs", due to
>> the configuration files being merged using the non-recursive //
>> operator. That's exactly why it's important to understand what the
>> needs are before we consider specific configuration mechanisms. It's
>> difficult to figure out if an idea will actually make things easier
>> without seeing an example of the problem, and what difference to it the
>> proposed solution would make.
>
> Well, the exact point of that is to give you a small next step to move
> towards a more flexible configuration system, without loosing control
> to revert back if the situation doesn't look better. It is unlikely a more
> complex contribution would be acceptable for you if even a little one
> like this wouldn't make it.
Well, it's not the size of the change that's important, but whether it
can be demonstrated that the change solves a problem. A big change to
fix a clear problem is fine!
But you've definitely pushed the conversation forward towards a more
flexible configuration system. In particular, we definitely need to
think better about custom VMs and especially optional host components,
and it seems to me that we need good, task-oriented documentation about
how to accomplish things that are already possible using the existing
mechanisms.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 13:22 ` Alyssa Ross
@ 2022-09-15 13:48 ` Ville Ilvonen
2022-09-15 14:00 ` Alyssa Ross
2022-09-16 4:59 ` [PATCH] Add image configuration option José Pekkarinen
1 sibling, 1 reply; 13+ messages in thread
From: Ville Ilvonen @ 2022-09-15 13:48 UTC (permalink / raw)
To: Alyssa Ross, José Pekkarinen; +Cc: devel
On 9/15/22 16:22, Alyssa Ross wrote:
> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>
>> On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>>
>>> [...]
>>> Okay, thanks for the explanation. I think we can group some of these
>>> together:
>>>
>>> • Stuff that's already Nixpkgs configuration options or can be
>>> expressed through an overlay. Whether to cross compile, what
>>> architecture to build for, whether to use a vendor kernel, etc. This
>>> can already be handled through the existing configuration mechanism.
>>>
>>> • VM customisation, including extra VMs, disabling Wayland, etc. In my
>>> mind there are still some open questions around how this should be
>>> implemented exactly, but this is definitely something that needs to
>>> be more configurable.
>>>
>>> • Whether to install extra stuff on the host system. This covers
>>> things like debugging symbols and tools.
>>>
>>> Does that sound right to you? Are there more that you can think of?
>>> I'd like to understand the requirements here better, to help think about
>>> what sort of configuration mechanisms might be required.
>>
>> This is a good summary, currently I can think of a coupel of more,
>> which is a mechanism to provide upstream project configuration artifacts
>> when nix allow to bypass their automated configs, for example, a kernel
>> config when using manual config kernel nix package, a bit more unclear
>> is how to provide upstream configs when nix is not flexible enough.
>
> You mean you'd like to manually provide a Kconfig file, rathen than
> using Nixpkgs' (not very good) structured config mechanism, right?
> That should be possible with an overlay, but maybe some documentation
> with an example would be a good idea?
>
>> A mechanism to generate a full image from the nix generated artifacts
>> putting together kernel, initrd, rootfs and ext partition so that the
>> full image can be flashed in a sdcard of choice and use it. This would
>> require to be configurable so that you can modify the partition table
>> to suit vendor needs.
>
> We've had some discussion about that already on the list and on IRC. My
> current view is that early boot firmware (U-Boot etc.) doesn't really
> have anything to do with Spectrum. They both run at different times,
> and they communicate over a standard interface (EBBR [1]), so the
> specifics of the firmware aren't really in scope for Spectrum itself and
> belong elsewhere. It doesn't make sense for Spectrum to be installing
> U-Boot any more than it makes sense for U-Boot to be installing
> Spectrum, or for Linux to be installing U-Boot — they are two separate
> components. (This isn't an approach unique to Spectrum — Fedora is
> doing something similar.)
>
> It can make sense to make an image that is a combination of U-Boot and
> Spectrum, but that process should be part of an integration between the
> two that exists one layer up, rather than part of either project. For
> example, you could do something like this:
>
> let
> spectrum = import <spectrum/img/live> {
> # config could either be loaded using the standard mechanism
> # or inlined here.
> };
> # It would also be possible to import individual components of
> # Spectrum and assemble them manually if even greater
> # flexibility was required, but I doubt that would be common.
>
> inherit (spectrum) pkgs;
> # I don't think a pkgs attribute currently exists on the
> # spectrum-live.img derivation, but it might make sense to add
> # for this sort of use case.
>
> uboot = pkgs.ubootRockPro64;
> in
>
> pkgs.runCommand "uboot+spectrum.img" {} ''
> # Use sfdisk (or maybe there's some better tool)
> # to create a partition table, and copy the U-boot image
> # and the Spectrum images into place.
> #
> # Spectrum is designed to accomodate this by not expecting
> # any of its partitions to be at any particular location
> # on disk.
> ''
>
> Maybe this is another case where documentation and a worked example
> would help?
Documented example case would help. It's good to scope but in the big
picture it's hard to see early boot firmware would have *nothing* to do
with Spectrum. That's not the case with x86_64 either.
Let me clarify.
On x86 traditionally people can't change early bootloading.
-> Spectrum assumes UEFI OS loading because UEFI is just there and can't
be changed
On arm traditionally people can and will change early bootloading.
-> Spectrum has assumed UEFI but UEFI is just not there. It's must be
put there - typically on device SD card or eMMC image.
On riscv assumption is more like on arm.
So the mechanism is essential, even when not *provided* by Spectrum it
should be acknowledged.
Documenting Spectrum reqs to boot itself with example determines how
easily people can make their devices run Spectrum.
-Ville
>
> [1]: https://arm-software.github.io/ebbr/index.html
>
>> [...]
>>> You're right — it's generally a bad thing if people have to patch
>>> Spectrum to make it fit their needs. I want to avoid that. But most of
>>> the things we've talked about so far don't feel to me like they're going
>>> to lead to massive configuration files. The exception is all the stuff
>>> that's either Nixpkgs configuration or overlays, but the mechanism you
>>> proposed here wouldn't help with that, because only a single
>>> configuration file would be able to change anything in "pkgs", due to
>>> the configuration files being merged using the non-recursive //
>>> operator. That's exactly why it's important to understand what the
>>> needs are before we consider specific configuration mechanisms. It's
>>> difficult to figure out if an idea will actually make things easier
>>> without seeing an example of the problem, and what difference to it the
>>> proposed solution would make.
>>
>> Well, the exact point of that is to give you a small next step to move
>> towards a more flexible configuration system, without loosing control
>> to revert back if the situation doesn't look better. It is unlikely a more
>> complex contribution would be acceptable for you if even a little one
>> like this wouldn't make it.
>
> Well, it's not the size of the change that's important, but whether it
> can be demonstrated that the change solves a problem. A big change to
> fix a clear problem is fine!
>
> But you've definitely pushed the conversation forward towards a more
> flexible configuration system. In particular, we definitely need to
> think better about custom VMs and especially optional host components,
> and it seems to me that we need good, task-oriented documentation about
> how to accomplish things that are already possible using the existing
> mechanisms.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 13:48 ` Ville Ilvonen
@ 2022-09-15 14:00 ` Alyssa Ross
2022-09-15 14:09 ` Ville Ilvonen
0 siblings, 1 reply; 13+ messages in thread
From: Alyssa Ross @ 2022-09-15 14:00 UTC (permalink / raw)
To: Ville Ilvonen, José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 6449 bytes --]
Ville Ilvonen <ville.ilvonen@unikie.com> writes:
> On 9/15/22 16:22, Alyssa Ross wrote:
>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>
>>> On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>>>
>>>> [...]
>>>> Okay, thanks for the explanation. I think we can group some of these
>>>> together:
>>>>
>>>> • Stuff that's already Nixpkgs configuration options or can be
>>>> expressed through an overlay. Whether to cross compile, what
>>>> architecture to build for, whether to use a vendor kernel, etc. This
>>>> can already be handled through the existing configuration mechanism.
>>>>
>>>> • VM customisation, including extra VMs, disabling Wayland, etc. In my
>>>> mind there are still some open questions around how this should be
>>>> implemented exactly, but this is definitely something that needs to
>>>> be more configurable.
>>>>
>>>> • Whether to install extra stuff on the host system. This covers
>>>> things like debugging symbols and tools.
>>>>
>>>> Does that sound right to you? Are there more that you can think of?
>>>> I'd like to understand the requirements here better, to help think about
>>>> what sort of configuration mechanisms might be required.
>>>
>>> This is a good summary, currently I can think of a coupel of more,
>>> which is a mechanism to provide upstream project configuration artifacts
>>> when nix allow to bypass their automated configs, for example, a kernel
>>> config when using manual config kernel nix package, a bit more unclear
>>> is how to provide upstream configs when nix is not flexible enough.
>>
>> You mean you'd like to manually provide a Kconfig file, rathen than
>> using Nixpkgs' (not very good) structured config mechanism, right?
>> That should be possible with an overlay, but maybe some documentation
>> with an example would be a good idea?
>>
>>> A mechanism to generate a full image from the nix generated artifacts
>>> putting together kernel, initrd, rootfs and ext partition so that the
>>> full image can be flashed in a sdcard of choice and use it. This would
>>> require to be configurable so that you can modify the partition table
>>> to suit vendor needs.
>>
>> We've had some discussion about that already on the list and on IRC. My
>> current view is that early boot firmware (U-Boot etc.) doesn't really
>> have anything to do with Spectrum. They both run at different times,
>> and they communicate over a standard interface (EBBR [1]), so the
>> specifics of the firmware aren't really in scope for Spectrum itself and
>> belong elsewhere. It doesn't make sense for Spectrum to be installing
>> U-Boot any more than it makes sense for U-Boot to be installing
>> Spectrum, or for Linux to be installing U-Boot — they are two separate
>> components. (This isn't an approach unique to Spectrum — Fedora is
>> doing something similar.)
>>
>> It can make sense to make an image that is a combination of U-Boot and
>> Spectrum, but that process should be part of an integration between the
>> two that exists one layer up, rather than part of either project. For
>> example, you could do something like this:
>>
>> let
>> spectrum = import <spectrum/img/live> {
>> # config could either be loaded using the standard mechanism
>> # or inlined here.
>> };
>> # It would also be possible to import individual components of
>> # Spectrum and assemble them manually if even greater
>> # flexibility was required, but I doubt that would be common.
>>
>> inherit (spectrum) pkgs;
>> # I don't think a pkgs attribute currently exists on the
>> # spectrum-live.img derivation, but it might make sense to add
>> # for this sort of use case.
>>
>> uboot = pkgs.ubootRockPro64;
>> in
>>
>> pkgs.runCommand "uboot+spectrum.img" {} ''
>> # Use sfdisk (or maybe there's some better tool)
>> # to create a partition table, and copy the U-boot image
>> # and the Spectrum images into place.
>> #
>> # Spectrum is designed to accomodate this by not expecting
>> # any of its partitions to be at any particular location
>> # on disk.
>> ''
>>
>> Maybe this is another case where documentation and a worked example
>> would help?
>
> Documented example case would help. It's good to scope but in the big
> picture it's hard to see early boot firmware would have *nothing* to do
> with Spectrum. That's not the case with x86_64 either.
>
> Let me clarify.
> On x86 traditionally people can't change early bootloading.
> -> Spectrum assumes UEFI OS loading because UEFI is just there and can't
> be changed
> On arm traditionally people can and will change early bootloading.
> -> Spectrum has assumed UEFI but UEFI is just not there. It's must be
> put there - typically on device SD card or eMMC image.
> On riscv assumption is more like on arm.
>
> So the mechanism is essential, even when not *provided* by Spectrum it
> should be acknowledged.
> Documenting Spectrum reqs to boot itself with example determines how
> easily people can make their devices run Spectrum.
Agreed, that's why I was pleased to discover the EBBR spec recently,
which defines exactly this: "an interface between platform firmware and
an operating system that is suitable for embedded platforms", designed
for U-Boot with UEFI like we were already targeting. So we can say
"Spectrum aims to implement EBBR on aarch64" (and on RISC-V when we get
there if that's the right thing to do), and that way there's a lovely
long document that explains what is Spectrum's responsibility to do, and
what is the firmware's responsibility to do. And when something goes
wrong, we'll be able to refer to the spec to determine whether it's a
problem with Spectrum, or with the platform firmware.
And of course we can have some documentation that introduces EBBR to an
audience that's not necessarily familiar with it, and provides an
example of how an EBBR system comprising both Spectrum and U-Boot might
be put together, expanding on what I included as the example in my
previous message. That should be more than enough to acknowledge the
mechanism, right?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 14:00 ` Alyssa Ross
@ 2022-09-15 14:09 ` Ville Ilvonen
2022-09-15 14:47 ` Integrating Spectrum and platform firmware Alyssa Ross
0 siblings, 1 reply; 13+ messages in thread
From: Ville Ilvonen @ 2022-09-15 14:09 UTC (permalink / raw)
To: Alyssa Ross, José Pekkarinen; +Cc: devel
On 9/15/22 17:00, Alyssa Ross wrote:
> Ville Ilvonen <ville.ilvonen@unikie.com> writes:
>
>> On 9/15/22 16:22, Alyssa Ross wrote:
>>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>>
>>>> On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>>>>
>>>>> [...]
>>>>> Okay, thanks for the explanation. I think we can group some of these
>>>>> together:
>>>>>
>>>>> • Stuff that's already Nixpkgs configuration options or can be
>>>>> expressed through an overlay. Whether to cross compile, what
>>>>> architecture to build for, whether to use a vendor kernel, etc. This
>>>>> can already be handled through the existing configuration mechanism.
>>>>>
>>>>> • VM customisation, including extra VMs, disabling Wayland, etc. In my
>>>>> mind there are still some open questions around how this should be
>>>>> implemented exactly, but this is definitely something that needs to
>>>>> be more configurable.
>>>>>
>>>>> • Whether to install extra stuff on the host system. This covers
>>>>> things like debugging symbols and tools.
>>>>>
>>>>> Does that sound right to you? Are there more that you can think of?
>>>>> I'd like to understand the requirements here better, to help think about
>>>>> what sort of configuration mechanisms might be required.
>>>>
>>>> This is a good summary, currently I can think of a coupel of more,
>>>> which is a mechanism to provide upstream project configuration artifacts
>>>> when nix allow to bypass their automated configs, for example, a kernel
>>>> config when using manual config kernel nix package, a bit more unclear
>>>> is how to provide upstream configs when nix is not flexible enough.
>>>
>>> You mean you'd like to manually provide a Kconfig file, rathen than
>>> using Nixpkgs' (not very good) structured config mechanism, right?
>>> That should be possible with an overlay, but maybe some documentation
>>> with an example would be a good idea?
>>>
>>>> A mechanism to generate a full image from the nix generated artifacts
>>>> putting together kernel, initrd, rootfs and ext partition so that the
>>>> full image can be flashed in a sdcard of choice and use it. This would
>>>> require to be configurable so that you can modify the partition table
>>>> to suit vendor needs.
>>>
>>> We've had some discussion about that already on the list and on IRC. My
>>> current view is that early boot firmware (U-Boot etc.) doesn't really
>>> have anything to do with Spectrum. They both run at different times,
>>> and they communicate over a standard interface (EBBR [1]), so the
>>> specifics of the firmware aren't really in scope for Spectrum itself and
>>> belong elsewhere. It doesn't make sense for Spectrum to be installing
>>> U-Boot any more than it makes sense for U-Boot to be installing
>>> Spectrum, or for Linux to be installing U-Boot — they are two separate
>>> components. (This isn't an approach unique to Spectrum — Fedora is
>>> doing something similar.)
>>>
>>> It can make sense to make an image that is a combination of U-Boot and
>>> Spectrum, but that process should be part of an integration between the
>>> two that exists one layer up, rather than part of either project. For
>>> example, you could do something like this:
>>>
>>> let
>>> spectrum = import <spectrum/img/live> {
>>> # config could either be loaded using the standard mechanism
>>> # or inlined here.
>>> };
>>> # It would also be possible to import individual components of
>>> # Spectrum and assemble them manually if even greater
>>> # flexibility was required, but I doubt that would be common.
>>>
>>> inherit (spectrum) pkgs;
>>> # I don't think a pkgs attribute currently exists on the
>>> # spectrum-live.img derivation, but it might make sense to add
>>> # for this sort of use case.
>>>
>>> uboot = pkgs.ubootRockPro64;
>>> in
>>>
>>> pkgs.runCommand "uboot+spectrum.img" {} ''
>>> # Use sfdisk (or maybe there's some better tool)
>>> # to create a partition table, and copy the U-boot image
>>> # and the Spectrum images into place.
>>> #
>>> # Spectrum is designed to accomodate this by not expecting
>>> # any of its partitions to be at any particular location
>>> # on disk.
>>> ''
>>>
>>> Maybe this is another case where documentation and a worked example
>>> would help?
>>
>> Documented example case would help. It's good to scope but in the big
>> picture it's hard to see early boot firmware would have *nothing* to do
>> with Spectrum. That's not the case with x86_64 either.
>>
>> Let me clarify.
>> On x86 traditionally people can't change early bootloading.
>> -> Spectrum assumes UEFI OS loading because UEFI is just there and can't
>> be changed
>> On arm traditionally people can and will change early bootloading.
>> -> Spectrum has assumed UEFI but UEFI is just not there. It's must be
>> put there - typically on device SD card or eMMC image.
>> On riscv assumption is more like on arm.
>>
>> So the mechanism is essential, even when not *provided* by Spectrum it
>> should be acknowledged.
>> Documenting Spectrum reqs to boot itself with example determines how
>> easily people can make their devices run Spectrum.
>
> Agreed, that's why I was pleased to discover the EBBR spec recently,
> which defines exactly this: "an interface between platform firmware and
> an operating system that is suitable for embedded platforms", designed
> for U-Boot with UEFI like we were already targeting. So we can say
> "Spectrum aims to implement EBBR on aarch64" (and on RISC-V when we get
> there if that's the right thing to do), and that way there's a lovely
> long document that explains what is Spectrum's responsibility to do, and
> what is the firmware's responsibility to do. And when something goes
> wrong, we'll be able to refer to the spec to determine whether it's a
> problem with Spectrum, or with the platform firmware.
>
> And of course we can have some documentation that introduces EBBR to an
> audience that's not necessarily familiar with it, and provides an
> example of how an EBBR system comprising both Spectrum and U-Boot might
> be put together, expanding on what I included as the example in my
> previous message. That should be more than enough to acknowledge the
> mechanism, right?
Example with some device(s) defines the usefulness - to get Spectrum
running on that device. Documentation with link to EBBR could be
additional reading. The last practical question is where the device
specific implementations of ebbr (e.g. u-boot) are stored. I'm reading
out of Spectrum tree but the "glue" nix (your example of uboot+spectrum)
would be needed somewhere. Could that be in Spectrum tree to be useful
for Spectrum users?
-Ville
^ permalink raw reply [flat|nested] 13+ messages in thread
* Integrating Spectrum and platform firmware
2022-09-15 14:09 ` Ville Ilvonen
@ 2022-09-15 14:47 ` Alyssa Ross
2022-09-16 5:29 ` Ville Ilvonen
0 siblings, 1 reply; 13+ messages in thread
From: Alyssa Ross @ 2022-09-15 14:47 UTC (permalink / raw)
To: Ville Ilvonen, José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 8075 bytes --]
Ville Ilvonen <ville.ilvonen@unikie.com> writes:
> On 9/15/22 17:00, Alyssa Ross wrote:
>> Ville Ilvonen <ville.ilvonen@unikie.com> writes:
>>
>>> On 9/15/22 16:22, Alyssa Ross wrote:
>>>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>>>
>>>>> A mechanism to generate a full image from the nix generated artifacts
>>>>> putting together kernel, initrd, rootfs and ext partition so that the
>>>>> full image can be flashed in a sdcard of choice and use it. This would
>>>>> require to be configurable so that you can modify the partition table
>>>>> to suit vendor needs.
>>>>
>>>> We've had some discussion about that already on the list and on IRC. My
>>>> current view is that early boot firmware (U-Boot etc.) doesn't really
>>>> have anything to do with Spectrum. They both run at different times,
>>>> and they communicate over a standard interface (EBBR [1]), so the
>>>> specifics of the firmware aren't really in scope for Spectrum itself and
>>>> belong elsewhere. It doesn't make sense for Spectrum to be installing
>>>> U-Boot any more than it makes sense for U-Boot to be installing
>>>> Spectrum, or for Linux to be installing U-Boot — they are two separate
>>>> components. (This isn't an approach unique to Spectrum — Fedora is
>>>> doing something similar.)
>>>>
>>>> It can make sense to make an image that is a combination of U-Boot and
>>>> Spectrum, but that process should be part of an integration between the
>>>> two that exists one layer up, rather than part of either project. For
>>>> example, you could do something like this:
>>>>
>>>> let
>>>> spectrum = import <spectrum/img/live> {
>>>> # config could either be loaded using the standard mechanism
>>>> # or inlined here.
>>>> };
>>>> # It would also be possible to import individual components of
>>>> # Spectrum and assemble them manually if even greater
>>>> # flexibility was required, but I doubt that would be common.
>>>>
>>>> inherit (spectrum) pkgs;
>>>> # I don't think a pkgs attribute currently exists on the
>>>> # spectrum-live.img derivation, but it might make sense to add
>>>> # for this sort of use case.
>>>>
>>>> uboot = pkgs.ubootRockPro64;
>>>> in
>>>>
>>>> pkgs.runCommand "uboot+spectrum.img" {} ''
>>>> # Use sfdisk (or maybe there's some better tool)
>>>> # to create a partition table, and copy the U-boot image
>>>> # and the Spectrum images into place.
>>>> #
>>>> # Spectrum is designed to accomodate this by not expecting
>>>> # any of its partitions to be at any particular location
>>>> # on disk.
>>>> ''
>>>>
>>>> Maybe this is another case where documentation and a worked example
>>>> would help?
>>>
>>> Documented example case would help. It's good to scope but in the big
>>> picture it's hard to see early boot firmware would have *nothing* to do
>>> with Spectrum. That's not the case with x86_64 either.
>>>
>>> Let me clarify.
>>> On x86 traditionally people can't change early bootloading.
>>> -> Spectrum assumes UEFI OS loading because UEFI is just there and can't
>>> be changed
>>> On arm traditionally people can and will change early bootloading.
>>> -> Spectrum has assumed UEFI but UEFI is just not there. It's must be
>>> put there - typically on device SD card or eMMC image.
>>> On riscv assumption is more like on arm.
>>>
>>> So the mechanism is essential, even when not *provided* by Spectrum it
>>> should be acknowledged.
>>> Documenting Spectrum reqs to boot itself with example determines how
>>> easily people can make their devices run Spectrum.
>>
>> Agreed, that's why I was pleased to discover the EBBR spec recently,
>> which defines exactly this: "an interface between platform firmware and
>> an operating system that is suitable for embedded platforms", designed
>> for U-Boot with UEFI like we were already targeting. So we can say
>> "Spectrum aims to implement EBBR on aarch64" (and on RISC-V when we get
>> there if that's the right thing to do), and that way there's a lovely
>> long document that explains what is Spectrum's responsibility to do, and
>> what is the firmware's responsibility to do. And when something goes
>> wrong, we'll be able to refer to the spec to determine whether it's a
>> problem with Spectrum, or with the platform firmware.
>>
>> And of course we can have some documentation that introduces EBBR to an
>> audience that's not necessarily familiar with it, and provides an
>> example of how an EBBR system comprising both Spectrum and U-Boot might
>> be put together, expanding on what I included as the example in my
>> previous message. That should be more than enough to acknowledge the
>> mechanism, right?
>
> Example with some device(s) defines the usefulness - to get Spectrum
> running on that device. Documentation with link to EBBR could be
> additional reading. The last practical question is where the device
> specific implementations of ebbr (e.g. u-boot) are stored. I'm reading
> out of Spectrum tree but the "glue" nix (your example of uboot+spectrum)
> would be needed somewhere. Could that be in Spectrum tree to be useful
> for Spectrum users?
Well, there are a couple of things here:
The first is that the glue Nix is only needed if you want to have the
firmware and the Spectrum partitions on the same image. This is
something that's supported, but recommended against (see the Firmware
Storage section of the EBBR spec [1] — that also mirrors the
recommendations I've heard from both the Tow-Boot and Fedora ARM
maintainers.)
That's not to say it's not a legitimate thing to do — I understand that
it's nice to be able to just download a single image and have everything
work, especially when Spectrum is one part of a bigger vision — so it's
not like it's unsupported, but the happy path that the community seems
to be heading towards (and therefore the one that I'd expect to
recommend to end users that are coming directly to Spectrum) is that
users should first install platform firmware if required (perhaps using
a distribution like Tow-Boot) onto dedicated storage as part of setting
up their device, and then install whatever EBBR-compliant distro they
want onto main storage. In the best case, where the hardware is well
supported by mainline Linux, they get a working device without needing
to track down or build a special image. If they're not quite so lucky,
they still don't need to worry about the specifics of combining firmware
and OS into a single image, and neither do we.
The second thing is that the Nix glue is board specific. It has to know
how to build U-Boot, and how to install it in the image at the correct
offset. But it isn't really specific to Spectrum at all. The only
interaction it needs to have with Spectrum is copying its partitions
into the right place, and the only implementation details of Spectrum it
needs to depend on are that Spectrum comes as a GPT image, and that it
doesn't mind if the offset of its partitions changes. So if you wanted
a reusable way to combine an OS image and a platform fimware image into
a single image with Nix, I think that would make more sense as a
seperate project outside of Spectrum, since it would also work with
almost any other OS.
I know there's been talk among NixOS aarch64 users/developers about
re-doing how NixOS builds SD card images, and this feels very relevant
to that conversation. This could, for example, be part of a future,
improved version of nixos-hardware, and as long as it didn't depend on
any NixOS specifics, it could work whether the OS being installed is
NixOS, Spectrum, Fedora, or anything else, as long as it implements
EBBR.
[1]: https://arm-software.github.io/ebbr/index.html#firmware-storage
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-15 13:22 ` Alyssa Ross
2022-09-15 13:48 ` Ville Ilvonen
@ 2022-09-16 4:59 ` José Pekkarinen
2022-09-16 7:25 ` Alyssa Ross
1 sibling, 1 reply; 13+ messages in thread
From: José Pekkarinen @ 2022-09-16 4:59 UTC (permalink / raw)
To: Alyssa Ross; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 2225 bytes --]
On Thu, Sep 15, 2022 at 4:22 PM Alyssa Ross <hi@alyssa.is> wrote:
> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>
> > On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>
> [...]
>
> You mean you'd like to manually provide a Kconfig file, rathen than
> using Nixpkgs' (not very good) structured config mechanism, right?
> That should be possible with an overlay, but maybe some documentation
> with an example would be a good idea?
>
Yes, but, for example, if I provide the overlay that uses that
that Kconfig, the Kconfig should be present in your system, as some
sort of default configuration for the developer to consume if they want
to use the overlay in question, otherwise, the developer needs to fetch
spectrum sources, and then fetch out the default configuration somewhere
else, put them together and test. The goal would be to upstream the overlay
so that one can take spectrum source code, make a config.nix to select
the overlay, and build, without extra steps to fetch other artifacts.
[...]
>
> Well, it's not the size of the change that's important, but whether it
> can be demonstrated that the change solves a problem. A big change to
> fix a clear problem is fine!
>
It is offering a way to template configurations for the cases
we were commenting before. So it solves a problem, the problem is
that currently the source code doesn't ship default configurations for
developers to test, so I can create a config.nix file downstream that
makes the overlay for the hardened kernel use case, and now, instead
of upstreaming and shipping it with any spectrum checkout, I have
to publish it somewhere else, and document how to put the puzzle
together so that a developer can test, use, and develop further. For
now, it doesn't let combine configuration files, so these templates
may be fat, because you can only make one template per case,
and choose it. In the future it would be good if they are small snippets
that do a particular purpose, and we list all the snippets we want
to make the full use case the user want(for ex. making a cross compiled
build from x86_64 of arm64 which includes security plus debugging).
José.
[-- Attachment #2: Type: text/html, Size: 3681 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Integrating Spectrum and platform firmware
2022-09-15 14:47 ` Integrating Spectrum and platform firmware Alyssa Ross
@ 2022-09-16 5:29 ` Ville Ilvonen
0 siblings, 0 replies; 13+ messages in thread
From: Ville Ilvonen @ 2022-09-16 5:29 UTC (permalink / raw)
To: Alyssa Ross, José Pekkarinen; +Cc: devel
On 9/15/22 17:47, Alyssa Ross wrote:
> Ville Ilvonen <ville.ilvonen@unikie.com> writes:
>
>> On 9/15/22 17:00, Alyssa Ross wrote:
>>> Ville Ilvonen <ville.ilvonen@unikie.com> writes:
>>>
>>>> On 9/15/22 16:22, Alyssa Ross wrote:
>>>>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>>>>
>>>>>> A mechanism to generate a full image from the nix generated artifacts
>>>>>> putting together kernel, initrd, rootfs and ext partition so that the
>>>>>> full image can be flashed in a sdcard of choice and use it. This would
>>>>>> require to be configurable so that you can modify the partition table
>>>>>> to suit vendor needs.
>>>>>
>>>>> We've had some discussion about that already on the list and on IRC. My
>>>>> current view is that early boot firmware (U-Boot etc.) doesn't really
>>>>> have anything to do with Spectrum. They both run at different times,
>>>>> and they communicate over a standard interface (EBBR [1]), so the
>>>>> specifics of the firmware aren't really in scope for Spectrum itself and
>>>>> belong elsewhere. It doesn't make sense for Spectrum to be installing
>>>>> U-Boot any more than it makes sense for U-Boot to be installing
>>>>> Spectrum, or for Linux to be installing U-Boot — they are two separate
>>>>> components. (This isn't an approach unique to Spectrum — Fedora is
>>>>> doing something similar.)
>>>>>
>>>>> It can make sense to make an image that is a combination of U-Boot and
>>>>> Spectrum, but that process should be part of an integration between the
>>>>> two that exists one layer up, rather than part of either project. For
>>>>> example, you could do something like this:
>>>>>
>>>>> let
>>>>> spectrum = import <spectrum/img/live> {
>>>>> # config could either be loaded using the standard mechanism
>>>>> # or inlined here.
>>>>> };
>>>>> # It would also be possible to import individual components of
>>>>> # Spectrum and assemble them manually if even greater
>>>>> # flexibility was required, but I doubt that would be common.
>>>>>
>>>>> inherit (spectrum) pkgs;
>>>>> # I don't think a pkgs attribute currently exists on the
>>>>> # spectrum-live.img derivation, but it might make sense to add
>>>>> # for this sort of use case.
>>>>>
>>>>> uboot = pkgs.ubootRockPro64;
>>>>> in
>>>>>
>>>>> pkgs.runCommand "uboot+spectrum.img" {} ''
>>>>> # Use sfdisk (or maybe there's some better tool)
>>>>> # to create a partition table, and copy the U-boot image
>>>>> # and the Spectrum images into place.
>>>>> #
>>>>> # Spectrum is designed to accomodate this by not expecting
>>>>> # any of its partitions to be at any particular location
>>>>> # on disk.
>>>>> ''
>>>>>
>>>>> Maybe this is another case where documentation and a worked example
>>>>> would help?
>>>>
>>>> Documented example case would help. It's good to scope but in the big
>>>> picture it's hard to see early boot firmware would have *nothing* to do
>>>> with Spectrum. That's not the case with x86_64 either.
>>>>
>>>> Let me clarify.
>>>> On x86 traditionally people can't change early bootloading.
>>>> -> Spectrum assumes UEFI OS loading because UEFI is just there and can't
>>>> be changed
>>>> On arm traditionally people can and will change early bootloading.
>>>> -> Spectrum has assumed UEFI but UEFI is just not there. It's must be
>>>> put there - typically on device SD card or eMMC image.
>>>> On riscv assumption is more like on arm.
>>>>
>>>> So the mechanism is essential, even when not *provided* by Spectrum it
>>>> should be acknowledged.
>>>> Documenting Spectrum reqs to boot itself with example determines how
>>>> easily people can make their devices run Spectrum.
>>>
>>> Agreed, that's why I was pleased to discover the EBBR spec recently,
>>> which defines exactly this: "an interface between platform firmware and
>>> an operating system that is suitable for embedded platforms", designed
>>> for U-Boot with UEFI like we were already targeting. So we can say
>>> "Spectrum aims to implement EBBR on aarch64" (and on RISC-V when we get
>>> there if that's the right thing to do), and that way there's a lovely
>>> long document that explains what is Spectrum's responsibility to do, and
>>> what is the firmware's responsibility to do. And when something goes
>>> wrong, we'll be able to refer to the spec to determine whether it's a
>>> problem with Spectrum, or with the platform firmware.
>>>
>>> And of course we can have some documentation that introduces EBBR to an
>>> audience that's not necessarily familiar with it, and provides an
>>> example of how an EBBR system comprising both Spectrum and U-Boot might
>>> be put together, expanding on what I included as the example in my
>>> previous message. That should be more than enough to acknowledge the
>>> mechanism, right?
>>
>> Example with some device(s) defines the usefulness - to get Spectrum
>> running on that device. Documentation with link to EBBR could be
>> additional reading. The last practical question is where the device
>> specific implementations of ebbr (e.g. u-boot) are stored. I'm reading
>> out of Spectrum tree but the "glue" nix (your example of uboot+spectrum)
>> would be needed somewhere. Could that be in Spectrum tree to be useful
>> for Spectrum users?
>
> Well, there are a couple of things here:
>
> The first is that the glue Nix is only needed if you want to have the
> firmware and the Spectrum partitions on the same image. This is
> something that's supported, but recommended against (see the Firmware
> Storage section of the EBBR spec [1] — that also mirrors the
> recommendations I've heard from both the Tow-Boot and Fedora ARM
> maintainers.)
Yes, I've read the spec and it's a good spec. They also stress it's not
an implementation. Technically the recommendation to store firmware
separately is sound. In practice, many ARM devices will require separate
OS to install/update the firmware WHEN OS chooses not to support it.
That is of course a design decision that pushes firmware install/update
out-of-tree. So having it that way not only leads to usability issues
for people who just want to try the OS but also makes development more
effort than it has to be. I really appreciate the Tow-Boot and Fedora
ARM effort. And I kind of like generic NixOS aarch64 image in its'
infancy. However, the ARM ecosystem constraints in device design and
vendor BSP support are beyond their control. Maybe more devices in
future will support vendor patched u-Boot in TowBoot package while still
being able to support booting OS installation from another media like
USB. As of today, that's not the case or is behind impractical
development effort.
>
> That's not to say it's not a legitimate thing to do — I understand that
> it's nice to be able to just download a single image and have everything
> work, especially when Spectrum is one part of a bigger vision — so it's
> not like it's unsupported, but the happy path that the community seems
> to be heading towards (and therefore the one that I'd expect to
> recommend to end users that are coming directly to Spectrum) is that
> users should first install platform firmware if required (perhaps using
> a distribution like Tow-Boot) onto dedicated storage as part of setting
> up their device, and then install whatever EBBR-compliant distro they
> want onto main storage. In the best case, where the hardware is well
> supported by mainline Linux, they get a working device without needing
> to track down or build a special image. If they're not quite so lucky,
> they still don't need to worry about the specifics of combining firmware
> and OS into a single image, and neither do we.
Sure. This is nice vision of future. We've now got mechanism to combine
FW and OS on single image for iMX8 dev boards - it's really user
friendly for those devices. And EBBR compliant. Breaking it down and
making separate installation processes/instructions for FW and OS maybe
possible in future but not priority as of now. This goes to the area of
bikeshedding vendor BSP supported mechanisms when what is needed is easy
install. Many other more relevant virtualization and security related
concepts to focus on first.
> The second thing is that the Nix glue is board specific. It has to know
> how to build U-Boot, and how to install it in the image at the correct
> offset. But it isn't really specific to Spectrum at all. The only
> interaction it needs to have with Spectrum is copying its partitions
> into the right place, and the only implementation details of Spectrum it
> needs to depend on are that Spectrum comes as a GPT image, and that it
> doesn't mind if the offset of its partitions changes. So if you wanted
> a reusable way to combine an OS image and a platform fimware image into
> a single image with Nix, I think that would make more sense as a
> seperate project outside of Spectrum, since it would also work with
> almost any other OS.
Ok. Out-of-tree it is then.
-Ville
>
> I know there's been talk among NixOS aarch64 users/developers about
> re-doing how NixOS builds SD card images, and this feels very relevant
> to that conversation. This could, for example, be part of a future,
> improved version of nixos-hardware, and as long as it didn't depend on
> any NixOS specifics, it could work whether the OS being installed is
> NixOS, Spectrum, Fedora, or anything else, as long as it implements
> EBBR.
>
> [1]: https://arm-software.github.io/ebbr/index.html#firmware-storage
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Add image configuration option
2022-09-16 4:59 ` [PATCH] Add image configuration option José Pekkarinen
@ 2022-09-16 7:25 ` Alyssa Ross
0 siblings, 0 replies; 13+ messages in thread
From: Alyssa Ross @ 2022-09-16 7:25 UTC (permalink / raw)
To: José Pekkarinen; +Cc: devel
[-- Attachment #1: Type: text/plain, Size: 4812 bytes --]
José Pekkarinen <jose.pekkarinen@unikie.com> writes:
> On Thu, Sep 15, 2022 at 4:22 PM Alyssa Ross <hi@alyssa.is> wrote:
>
>> José Pekkarinen <jose.pekkarinen@unikie.com> writes:
>>
>> > On Thu, Sep 15, 2022 at 2:31 PM Alyssa Ross <hi@alyssa.is> wrote:
>>
>> [...]
>>
>> You mean you'd like to manually provide a Kconfig file, rathen than
>> using Nixpkgs' (not very good) structured config mechanism, right?
>> That should be possible with an overlay, but maybe some documentation
>> with an example would be a good idea?
>>
>
> Yes, but, for example, if I provide the overlay that uses that
> that Kconfig, the Kconfig should be present in your system, as some
> sort of default configuration for the developer to consume if they want
> to use the overlay in question, otherwise, the developer needs to fetch
> spectrum sources, and then fetch out the default configuration somewhere
> else, put them together and test. The goal would be to upstream the overlay
> so that one can take spectrum source code, make a config.nix to select
> the overlay, and build, without extra steps to fetch other artifacts.
>
> [...]
>>
>> Well, it's not the size of the change that's important, but whether it
>> can be demonstrated that the change solves a problem. A big change to
>> fix a clear problem is fine!
>>
>
> It is offering a way to template configurations for the cases
> we were commenting before. So it solves a problem, the problem is
> that currently the source code doesn't ship default configurations for
> developers to test, so I can create a config.nix file downstream that
> makes the overlay for the hardened kernel use case, and now, instead
> of upstreaming and shipping it with any spectrum checkout, I have
> to publish it somewhere else, and document how to put the puzzle
> together so that a developer can test, use, and develop further. For
> now, it doesn't let combine configuration files, so these templates
> may be fat, because you can only make one template per case,
> and choose it. In the future it would be good if they are small snippets
> that do a particular purpose, and we list all the snippets we want
> to make the full use case the user want(for ex. making a cross compiled
> build from x86_64 of arm64 which includes security plus debugging).
Well, it's not true that it doesn't let you combine files — it lets you
do that just as much as your patch would (and more flexibly). For
example:
# config1.nix
import ./config2.nix // {
# other config goes here
}
# config2.nix
{
pkgs = ...;
}
But I understand what you're saying about wanting to keep reusable bits
of configuration in the Spectrum repository. I think it would be
reasonable to have a "contrib" directory, where extra stuff (like
reusable config snippets) that aren't part of the default system but
might be useful to people working with it could live, as long as
everything required to build a system with that configuration was
already present and working in Spectrum so it could be tested (so no
aarch64 configs before we have aarch64 support, for example).
It's also going to be worth considering when it just makes more sense
for things to be first class options instead of reusable configuration
modules. Those would be easier to use in a configuration file, but if
we end up with too many of them I think it'll be a maintainability
problem.
One thing I'm already thinking is that maybe it would be an improvement
to allow setting individual Nixpkgs options in the configuration file,
and have the import handled by Spectrum by default, rather than making
the configuration file do the pkgs import as we currently do. Because
if the configuration file has to import pkgs, that prevents the Spectrum
build system from setting any configuration of its own. That prevents
us doing smart things that I think we'd like to do at some point.
For example, it would be nice at some point to support cross-compiling
from non-Linux machines. And in that case, it would be nice if Spectrum
was automatically cross-compiled, for Linux on whatever architecture the
build machine is using by default. That would look something like:
let
splitCurrentSystem = builtins.split "-" builtins.currentSystem;
currentArch = builtins.head splitCurrentSystem;
currentKernel = builtins.tail splitCurrentSystem;
in
import <nixpkgs> ((if currentKernel != "linux" then {
crossSystem.system = "${currentArch}-linux";
} else {}) // {
# other config
})
Which would be unacceptably horrible to make people specify in their
configuration files when it's something the build system could be doing
for them automatically.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2022-09-16 7:26 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-15 7:35 [PATCH] Add image configuration option José Pekkarinen
2022-09-15 8:21 ` Alyssa Ross
2022-09-15 10:42 ` José Pekkarinen
2022-09-15 11:31 ` Alyssa Ross
2022-09-15 12:31 ` José Pekkarinen
2022-09-15 13:22 ` Alyssa Ross
2022-09-15 13:48 ` Ville Ilvonen
2022-09-15 14:00 ` Alyssa Ross
2022-09-15 14:09 ` Ville Ilvonen
2022-09-15 14:47 ` Integrating Spectrum and platform firmware Alyssa Ross
2022-09-16 5:29 ` Ville Ilvonen
2022-09-16 4:59 ` [PATCH] Add image configuration option José Pekkarinen
2022-09-16 7:25 ` Alyssa Ross
Code repositories for project(s) associated with this public inbox
https://spectrum-os.org/git/crosvm
https://spectrum-os.org/git/doc
https://spectrum-os.org/git/mktuntap
https://spectrum-os.org/git/nixpkgs
https://spectrum-os.org/git/spectrum
https://spectrum-os.org/git/ucspi-vsock
https://spectrum-os.org/git/www
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).