From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id EA69BBC6A7; Sat, 27 May 2023 15:07:36 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id 19869BC60F; Sat, 27 May 2023 15:07:34 +0000 (UTC) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by atuin.qyliss.net (Postfix) with ESMTPS id 081F2BC68B for ; Sat, 27 May 2023 15:07:31 +0000 (UTC) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 3943D5C00B0; Sat, 27 May 2023 11:07:31 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Sat, 27 May 2023 11:07:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1685200051; x=1685286451; bh=0W CGfLTVAHmDbxsDiXxa3ocsFh4VpAKNdCcRdTTM+AQ=; b=IKQmBolGO7Db+sBxMR 6mIAOXGMYW6PiLAFL2gpa/ibQ24nkrhxJpXC9wNs48M8Xs/Ha8PtwijIvAYSRVCe qqYXKRg0kQnrZo3webi+GJ6DBjCTchxLLTp2720RjGClOLRQKX7uyFwI85lfekqT YsFpMSTT+AihjEzZePo7rlKpsZyuD4IyWW6eYS383B2HJk8JVcIHhmsYpy/BO5lD 4+ZvMfICPGnLTQY1RD0ncn/Sp2w7rzag0effnABnBNqLNYv0mqIhfcmurr4QhkSZ hyCz5roNd6Kt04xGEDM1ojKz+jHUO8VFDnm1BEC3nwj0EtvzeeeEEwKuVOCyRRB3 UECw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1685200051; x=1685286451; bh=0WCGfLTVAHmDb xsDiXxa3ocsFh4VpAKNdCcRdTTM+AQ=; b=oGlr+ijndy/VCnbkOT+vLE0LCEyzM ANe1wzMHl2wlpkrEuCgz0e+Ck1JTpcMwikZGnhQ+ItJ8TWyH/kqvuOQEYqrb2PT0 ol76A3MtxaFoASg8ndFG9rUqhK4KoMZQFcmcgCN/zVP20a7NtIqqYiJSC1Lm9+Qs Z7edPKc+Tq89l0b/nBbYYRjem5VTfwXiyE89T8y/wPFIcRH8ixLMM1zXz40+yBd0 YIT/c3vbVxsfRXFOMXHwPuJTpvXLeAtXFfmPK7WCwtw1tXBhkwzEWMVOf5vVVs+k YehKHsbzKr/dB2F6HBYiH3R1FVJzTTFQ8qEzB2JMeFG64J48MqDoE7S0g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfeekuddgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne gfrhhlucfvnfffucdlvdefmdenucfjughrpefhvfevufgjfhffkfggtgesghdtreertddt tdenucfhrhhomheptehlhihsshgrucftohhsshcuoehhihesrghlhihsshgrrdhisheqne cuggftrfgrthhtvghrnhephefhtdffueetieffkeejgfeggeetkeduieefjeeuiefftddt fedvgfevheetudeknecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehhihesrghlhihsshgrrdhi sh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 27 May 2023 11:07:30 -0400 (EDT) Received: by x220.qyliss.net (Postfix, from userid 1000) id 2EB0054B4; Sat, 27 May 2023 15:07:29 +0000 (UTC) From: Alyssa Ross To: Ryan Lahfa Subject: Re: [PATCH] release/checks/try.nix: init In-Reply-To: References: <20230526210757.397735-1-hi@alyssa.is> <87353halgh.fsf@alyssa.is> Date: Sat, 27 May 2023 15:07:20 +0000 Message-ID: <87y1l995jr.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Message-ID-Hash: W7IGBT2QCVZC3GUU3SIDOWETZFWOUE4M X-Message-ID-Hash: W7IGBT2QCVZC3GUU3SIDOWETZFWOUE4M X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: devel@spectrum-os.org X-Mailman-Version: 3.3.5 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Ryan Lahfa writes: > On Sat, May 27, 2023 at 02:38:22PM +0000, Alyssa Ross wrote: >> Ryan Lahfa writes: >>=20 >> > On Fri, May 26, 2023 at 09:07:58PM +0000, Alyssa Ross wrote: >> >> This is a regression test for c7f87f3 ("host/rootfs: allow growing ext >> >> partition to fail"). It's the first time we're actually doing >> >> automated tests of a Spectrum boot. For now, I'm using the NixOS test >> >> framework, but because we're not using NixOS and not setting any NixOS >> >> options, it feels to me like it doesn't actually buy us very much, so >> >> if it doesn't start adding more value as we add more (or more complex) >> >> tests, it might be simpler to just use a shell/execline script for >> >> tests. >> > >> > Are you only interested into tests that works without any >> > instrumentation? >> > >> > e.g. what if Spectrum added the backdoor.service in their service >> > management? That'd repair the ability of the test framework to have >> > better interactions with the guest without QEMU Agent. >>=20 >> At least until I really need tests that depend on guest cooperation, yes. >> Because to have either the backdoor service or the QEMU agent, I'd >> either have to build custom images just for testing (which would mean >> the real images were not actually tested), or I'd have to build those >> things into all Spectrum images. >>=20 >> At some point, it might not be possible to get away from this, but the >> basic tests I've written so far have all gone fine without the need for >> any guest cooperation beyond a serial shell. > > Makes sense, supporting "minimal" friction for this usecase is > desirable. > > I guess we could have Python *and* Bash/execline as "test scripts" if we = want. > > But I would need to see more about what would Bash/execline-oriented > test scripts would look like in practice. > > Python is suboptimal for various things but the "batteries included" are > very useful for getting a OK-grade thing running. I don't think there's much of a reason to support shell/execline scripts for NixOS tests. My point here is more that I'm not really getting anything out of the NixOS test setup at all. I already have to construct my own QEMU command line, so the only thing I'm really getting out of it is the wait_for_console_text() function, which wouldn't be _that_ hard to do without in a shell/execline script either. It would probably be different if I _was_ using a guest agent / backdoor service, so I do think there's value to the NixOS test framework trying to support other distros. It would be useful when the tests would benefit from its built-in systemd integration, or OCR functionality, etc. >> >> +config.pkgs.nixosTest ({ pkgs, ... }: { >> >> + name =3D "try-spectrum-test"; >> >> + nodes =3D {}; >> >> + >> >> + testScript =3D '' >> >> + import shlex >> >> + import subprocess >> >> + >> >> + conf =3D subprocess.run([ >> >> + "${pkgs.mtools}/bin/mcopy", >> >> + "-i", >> >> + "${live}@@1M", >> >> + "::loader/entries/spectrum.conf", >> >> + "-", >> >> + ], stdout=3Dsubprocess.PIPE) >> >> + conf.check_returncode() >> >> + >> >> + cmdline =3D None >> >> + for line in conf.stdout.decode('utf-8').splitlines(): >> >> + key, value =3D line.split(' ', 1) >> >> + if key =3D=3D 'options': >> >> + cmdline =3D value >> > >> > Is there any reason to not have `conf` / `cmdline` >> > being derived in a derivation? >>=20 >> We can't know it at eval time, because it includes the verity hash. So >> our options are to recompute it here, or reuse the results of the >> derivation that already computed it, which is the approach I've taken >> here. It's a bit unweildy, but I blame Python for that. If we do end >> up switching to shell it'd just be: >>=20 >> mcopy -i ${live}@@1M ::loader/entries/spectrum.conf - | sed -n 's/^opti= ons //p' > > Makes sense, but do you need it at evaltime? > > Wouldn't this be a possibility: > > ```nix > let optionsFile =3D pkgs.runCommand "extract-options" {} ''mcopy -i ${liv= e}@@1M ::loader/entries/spectrum.conf - | sed -n 's/^options //p' > $out''; > in > '' > with open(${optionsFile}, "r") as f: > options =3D f.read() > '' > ``` > > Ideally, with the https://github.com/NixOS/nixpkgs/issues/156563 > proposal, this could even become easier IMHO. Well yeah, I could, but I think that's just extra complexity over doing it in the driver script. It's not _that_ bad in Python. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEH9wgcxqlHM/ARR3h+dvtSFmyccAFAmRyHKgACgkQ+dvtSFmy ccA00A//bUlJCjuczxus/sy0tbUOIzS/N2Q5T4ur9yntcFydMvegpNGdc9oDcywD 84OORm6epQNzVJLxfmSMVCYojgzP6e3MDEhKfKik9Fl00VoRZs23UkByGo/hsWA4 2Pw3yj+Jo96ZCdKjbK9XUaA1yIPOMdH4FqdW6tuTbzmuiCDyvI7EDrg57sJtFjQj a6I+insI2qNV1G2yb9kvvLBhT4A2HqyHreOknj7WWm98AvVE5kGLJxFflA4aUU/i VNwiEcL+EgPnFE3IdoCt2JTyek5nG6u6cTISRZJoochjVKF/x2qPDkTlZIo2uEw8 JgU6HLkmG8NkStGKhQ59y10FbSRNACSnI2st6Ghj+tGMdNeE0sw1UdO1yLcXB+jp JgqGiY4dBUOdhKcmktY706NTY7/yq17dULmh/rA3ArP/7SJKVdhAdEIigoCzC3ed n0d56FSJ0UmsWhC7XIHOxbvegZfgHkxX4M1sX1Fjx61VFddi8bDQ1rENUe511AmI upgdRXCwqgoKFTUNrlaTWTM7qU3JHQFPwTep9Rw6qC06ICHwi4sPclnsprqurFfo z/wiEv/jQICstdgLa412cY70YjUIdP/N/Fou3BtnMiPth00aNBz05DQYyb5HH1p9 B8ndV/meVgJH8+MwbaV7FGYrDAIlFhrUUhExmEEYTL7VsDy3FKs= =ns5K -----END PGP SIGNATURE----- --=-=-=--