From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 8BEF221098; Wed, 18 May 2022 08:55:44 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id 0282B21056; Wed, 18 May 2022 08:55:42 +0000 (UTC) Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by atuin.qyliss.net (Postfix) with ESMTPS id 2B6CB21052 for ; Wed, 18 May 2022 08:55:38 +0000 (UTC) Received: by mail-oi1-x233.google.com with SMTP id j12so1937093oie.1 for ; Wed, 18 May 2022 01:55:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0ZsjXYACDcsjSBTC9s0sLdgKA4qnIE282Hvs2NJ/4x4=; b=h4r0wcwRgHrpsjo2AQIEY1TohVzoLGY6lU07WyQ+Vz0Ymks05bRkMt2/HeIP3e/1rm JjNRRZ+VDWBsnI0qKZqp0zOgO6O3Roncd0s1MjhIvAnQZVnZmM/AJP2hSgsBAziG6q4p tN0gWR+mTntFL7jWdIUxDWBSRVc5sV04OIbJGmXdj8LAcxyhApBWJDxDd5h/B7JvQFk5 T0zXjLGajLEVaSzIq+bcTaFoesHbW64dpVwNUWKzBzApece7uihIKTNWqKIRr/caZ3b8 Q2Mmhk5VVUWAqIE7XrqpKV/nIzxkOmNPMUP7TOAZs3c9S3Jw+Crp2DkZVII/x+tGRGGU XS1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0ZsjXYACDcsjSBTC9s0sLdgKA4qnIE282Hvs2NJ/4x4=; b=MA5mO1oY9UAkX30pCTStWz3s3kkLmYYV1fLXMyNkESNsCewoVpH5Kh8OTVqu43DzL8 g6Sc1G/dQv1hVVFW5D9amhsKP7Qh0I96fxMFiV7L4mKtm0MCIPMRbAcvnvrQp7gJEug9 ztnSPg9/PnBQnf0V9ZeF1TxSKu12FlI6SH7XHktJsYECt5TyfrmNWly1kH9Vx+RvcUMq wciAlzSAw7Wx7hydfVelf6b/wHQ1PvwrqGkUwNukjRebsLPiS40zMc3fDRrttahz7dw2 36hmMiiU3XJMjFFJfiZW/i4P+41O/35Pdu3Nd/3riAQhYE7iRkzHl9C49DMMBLIiAom+ TVDw== X-Gm-Message-State: AOAM532U+fvuM4gB3SjKIViyn9qYYUZHRjWMSMzNfXo8PJOfdXUCOW+l +/gfg2wkx7eUVBJ2jEzfrBUvespYL49/IcTgpvw= X-Google-Smtp-Source: ABdhPJwMxi4qHl3wRbyhGd9CaZ3GjvVb1Wu649VIqbQkGRKGIhhIfa0eggJ71dRWrWb/756em5dd61WbSPUBGHtvdI4= X-Received: by 2002:a05:6808:3098:b0:326:7275:73ec with SMTP id bl24-20020a056808309800b00326727573ecmr12082265oib.54.1652864134797; Wed, 18 May 2022 01:55:34 -0700 (PDT) MIME-Version: 1.0 References: <20220321160534.w4oifq3cdygrobrm@eve> <20220322111639.pvr5gxz6zmeu5oua@eve> <20220414135737.lvfbrch34kjojn3s@eve> <20220516115527.2a3wnqqgzuo7gt3x@eve> In-Reply-To: <20220516115527.2a3wnqqgzuo7gt3x@eve> From: Thomas Leonard Date: Wed, 18 May 2022 09:55:21 +0000 Message-ID: Subject: Re: Using virtio-gpu instead of virtwl To: Alyssa Ross Content-Type: multipart/mixed; boundary="0000000000009e22c005df456bb6" Message-ID-Hash: SVQTZYHCGRFOBYZJPPSVOAWGLBIM6NPI X-Message-ID-Hash: SVQTZYHCGRFOBYZJPPSVOAWGLBIM6NPI X-MailFrom: talex5@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; header-match-discuss.spectrum-os.org-0; header-match-discuss.spectrum-os.org-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: discuss@spectrum-os.org X-Mailman-Version: 3.3.5 Precedence: list List-Id: General high-level discussion about Spectrum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --0000000000009e22c005df456bb6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 16 May 2022 at 11:55, Alyssa Ross wrote: > > Thanks for the update! > > On Sun, May 15, 2022 at 03:20:24PM +0000, Thomas Leonard wrote: > > On Thu, 14 Apr 2022 at 13:57, Alyssa Ross wrote: > > > > > > On Wed, Apr 13, 2022 at 05:12:13PM +0000, Thomas Leonard wrote: > > > > On Wed, 6 Apr 2022 at 12:19, Thomas Leonard wrot= e: > > > > [ converting from virtwl to virtio-gpu ] > > > > > I tried, but failed, to figure out the protocol. I did manage to = get a > > > > > test application showing a little animation, but it crashes after= a > > > > > few seconds. > > > > > > > > OK, I found a solution to this: you can just open the device file > > > > twice and use one instance for Wayland messages and the other for > > > > allocating images. This avoids the first race. With that, I got the > > > > proxy converted: > > > > > > > > https://github.com/talex5/wayland-proxy-virtwl/pull/28 > > > > > > > > Though I'm not sure it's an improvement: +1,819 =E2=88=92577 lines! > > > > > > > > Instructions for configuring crosvm to use it: > > > > > > > > https://github.com/talex5/wayland-proxy-virtwl#virtio-gpu-support > > > > > > > > And I wrote up my guesses about the protocol here: > > > > > > > > https://github.com/talex5/wayland-proxy-virtwl/blob/master/virtio= -spec.md > > > > > > That's extremely helpful, thanks for writing it up! > > > > A small update on this: > > > > First, I got virtio-gpu working with the jail. It just needs a couple > > of extra paths for NixOS: > > > > diff --git a/src/linux.rs b/src/linux.rs > > index ad031749..52d3142f 100644 > > --- a/src/linux.rs > > +++ b/src/linux.rs > > @@ -790,6 +790,8 @@ fn gpu_jail(cfg: &Config, policy: &str) -> > > Result> { > > jail_mount_bind_if_exists( > > &mut jail, > > &[ > > + "/run/opengl-driver", > > + "/nix/store", > > "/usr/lib", > > "/usr/lib64", > > "/lib", > > > > Secondly, I realised that the "video" memory returned by virtio-gpu > > was just regular host memory (from stracing crosvm). This is because > > crosvm is compiled without minigbm support and falls back to this. > > > > After enabling minigbm and its amdgpu support (which also required > > adding a dependency on mesa) it started allocating vram. However, this > > broke the proxy because vram can't be used with the Wl_shm protocol. > > Do you have a Nix expression somewhere for crosvm with all this stuff > fixed? I'd like to integrate it into my draft Nixpkgs PR. (If you > didn't do it with Nix, I can make the changes myself.) No, sorry. I've just been hacking on it in nix-shell. The only change I made to nixpkgs was adding mesa as a dependency. The changes to get minigbm to compile are just a hack for my system. But I've attached the patches just for reference. However, since it stops the proxy from working, these patches currently aren't an improvement from my point of view! BTW, I did get a little further: using __builtin_ia32_clflush fixed the cache-coherency problem and the scrolling pattern now looks correct. However, it still crashes the VM if you resize the window a bit. I modified crosvm to clear the vram before handing it to the guest (it wasn't doing that before), and that doesn't crash, so the memory is mapped correctly on the host side. --=20 talex5 (GitHub/Twitter) http://roscidus.com/blog/ --0000000000009e22c005df456bb6 Content-Type: application/octet-stream; name="mesa.patch" Content-Disposition: attachment; filename="mesa.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l3beh4tm0 aW5kZXggMjZlODllMGNmMjAuLjA2YmFmMGJiMzUyIDEwMDY0NAotLS0gYS9wa2dzL2FwcGxpY2F0 aW9ucy92aXJ0dWFsaXphdGlvbi9jcm9zdm0vZGVmYXVsdC5uaXgKKysrIGIvcGtncy9hcHBsaWNh dGlvbnMvdmlydHVhbGl6YXRpb24vY3Jvc3ZtL2RlZmF1bHQubml4CkBAIC0xLDcgKzEsNyBAQAog eyBzdGRlbnYsIGxpYiwgcnVzdCwgcnVzdFBsYXRmb3JtLCBmZXRjaGdpdAogLCBtZXNvbiwgbmlu amEsIHBrZy1jb25maWcsIHB5dGhvbjMsIHdheWxhbmQtc2Nhbm5lcgogLCBsaWJjYXAsIGxpYmRy bSwgbGliZXBveHksIGxpYmdsdm5kLCBtaW5pamFpbCwgd2F5bGFuZCwgd2F5bGFuZC1wcm90b2Nv bHMsIHhvcmcKLSwgbGludXgKKywgbGludXgsIG1lc2EKIH06CiAKIGxldApAQCAtNDAsNyArNDAs NyBAQCBpbgogCiAgICAgYnVpbGRJbnB1dHMgPSBbCiAgICAgICBsaWJjYXAgbGliZHJtIGxpYmVw b3h5IGxpYmdsdm5kIG1pbmlqYWlsIHdheWxhbmQgd2F5bGFuZC1wcm90b2NvbHMKLSAgICAgIHhv cmcubGliWDExCisgICAgICB4b3JnLmxpYlgxMSBtZXNhCiAgICAgXTsKIAogICAgIHBvc3RQYXRj aCA9ICcnCg== --0000000000009e22c005df456bb6 Content-Type: application/octet-stream; name="minigbm.patch" Content-Disposition: attachment; filename="minigbm.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l3behb3e1 ZGlmZiAtLWdpdCBhL01ha2VmaWxlIGIvTWFrZWZpbGUKaW5kZXggOTg3NzA4Zi4uOTQ0MjkxMSAx MDA2NDQKLS0tIGEvTWFrZWZpbGUKKysrIGIvTWFrZWZpbGUKQEAgLTEwLDEwICsxMCwxMCBAQCBQ Q19MSUJTIDo9ICQoc2hlbGwgJChQS0dfQ09ORklHKSAtLWxpYnMgJChQQ19ERVBTKSkKIAogQ1BQ RkxBR1MgKz0gLURfR05VX1NPVVJDRT0xCiBDRkxBR1MgKz0gLXN0ZD1jOTkgLVdhbGwgLVdzaWdu LWNvbXBhcmUgLVdwb2ludGVyLWFyaXRoIC1XY2FzdC1xdWFsIFwKLSAgICAgICAgIC1XY2FzdC1h bGlnbiAtRF9HTlVfU09VUkNFPTEgLURfRklMRV9PRkZTRVRfQklUUz02NAorICAgICAgICAgLVdj YXN0LWFsaWduIC1EX0dOVV9TT1VSQ0U9MSAtRF9GSUxFX09GRlNFVF9CSVRTPTY0IC1ERFJWX0FN REdQVSAtRERSSV9EUklWRVJfRElSPS9ydW4vb3BlbmdsLWRyaXZlci9saWIvZHJpCiAKIGlmZGVm IERSVl9BTURHUFUKLSAgICAgICBDRkxBR1MgKz0gJChzaGVsbCAkKFBLR19DT05GSUcpIC0tY2Zs YWdzIGxpYmRybV9hbWRncHUpCisgICAgICAgQ0ZMQUdTICs9ICQoc2hlbGwgJChQS0dfQ09ORklH KSAtLWNmbGFncyBsaWJkcm1fYW1kZ3B1IG9zbWVzYSkKICAgICAgICBMRExJQlMgKz0gLWxkcm1f YW1kZ3B1IC1sZGwKIGVuZGlmCiBpZmRlZiBEUlZfSTkxNQpkaWZmIC0tZ2l0IGEvZHJpLmMgYi9k cmkuYwppbmRleCA4YjU1YzMyLi5hYzg3OTRiIDEwMDY0NAotLS0gYS9kcmkuYworKysgYi9kcmku YwpAQCAtNDUyLDcgKzQ1Miw3IEBAIHNpemVfdCBkcmlfbnVtX3BsYW5lc19mcm9tX21vZGlmaWVy KHN0cnVjdCBkcml2ZXIgKmRydiwgdWludDMyX3QgZm9ybWF0LCB1aW50NjRfCiAgICAgICAgfQog CiAgICAgICAgdWludDY0X3QgcGxhbmVzOwotICAgICAgIEdMYm9vbGVhbiByZXQgPSBkcmktPmlt YWdlX2V4dGVuc2lvbi0+cXVlcnlEbWFCdWZGb3JtYXRNb2RpZmllckF0dHJpYnMoCisgICAgICAg aW50IHJldCA9IGRyaS0+aW1hZ2VfZXh0ZW5zaW9uLT5xdWVyeURtYUJ1ZkZvcm1hdE1vZGlmaWVy QXR0cmlicygKICAgICAgICAgICAgZHJpLT5kZXZpY2UsIGZvcm1hdCwgbW9kaWZpZXIsIF9fRFJJ X0lNQUdFX0ZPUk1BVF9NT0RJRklFUl9BVFRSSUJfUExBTkVfQ09VTlQsICZwbGFuZXMpOwogICAg ICAgIGlmICghcmV0KQogICAgICAgICAgICAgICAgcmV0dXJuIDA7Cg== --0000000000009e22c005df456bb6 Content-Type: application/octet-stream; name="crosvm.patch" Content-Disposition: attachment; filename="crosvm.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l3bejnd22 ZGlmZiAtLWdpdCBhL0NhcmdvLnRvbWwgYi9DYXJnby50b21sCmluZGV4IGU5OGM3ZGFhLi44YzFk ZmNhMCAxMDA2NDQKLS0tIGEvQ2FyZ28udG9tbAorKysgYi9DYXJnby50b21sCkBAIC0xMzQsNyAr MTM0LDcgQEAgZGlzayA9IHsgcGF0aCA9ICJkaXNrIiB9CiBlbnVtbiA9ICIwLjEuMCIKIGdkYnN0 dWIgPSB7IHZlcnNpb24gPSAiMC41LjAiLCBvcHRpb25hbCA9IHRydWUgfQogZ2Ric3R1Yl9hcmNo ID0geyB2ZXJzaW9uID0gIjAuMS4wIiwgb3B0aW9uYWwgPSB0cnVlIH0KLXJ1dGFiYWdhX2dmeCA9 IHsgcGF0aCA9ICJydXRhYmFnYV9nZngifQorcnV0YWJhZ2FfZ2Z4ID0geyBwYXRoID0gInJ1dGFi YWdhX2dmeCIsIGZlYXR1cmVzID0gWyJtaW5pZ2JtIl19CiBoeXBlcnZpc29yID0geyBwYXRoID0g Imh5cGVydmlzb3IiIH0KIGtlcm5lbF9jbWRsaW5lID0geyBwYXRoID0gImtlcm5lbF9jbWRsaW5l IiB9CiBrZXJuZWxfbG9hZGVyID0geyBwYXRoID0gImtlcm5lbF9sb2FkZXIiIH0KZGlmZiAtLWdp dCBhL3J1dGFiYWdhX2dmeC9idWlsZC5ycyBiL3J1dGFiYWdhX2dmeC9idWlsZC5ycwppbmRleCBl YjEwOWEyOS4uMzY2ZDg5NTMgMTAwNjQ0Ci0tLSBhL3J1dGFiYWdhX2dmeC9idWlsZC5ycworKysg Yi9ydXRhYmFnYV9nZngvYnVpbGQucnMKQEAgLTcyLDYgKzcyLDcgQEAgZm4gYnVpbGRfbWluaWdi bShvdXRfZGlyOiAmUGF0aCkgLT4gUmVzdWx0PCgpPiB7CiAKICAgICBsZXQgbWFrZV9mbGFncyA9 IGVudjo6dmFyKCJDQVJHT19NQUtFRkxBR1MiKS51bndyYXAoKTsKICAgICBsZXQgc3RhdHVzID0g Q29tbWFuZDo6bmV3KCJtYWtlIikKKyAgICAgICAgLmVudigiRFJWX0FNREdQVSIsICIxIikKICAg ICAgICAgLmVudigiTUFLRUZMQUdTIiwgbWFrZV9mbGFncykKICAgICAgICAgLmVudigiQ1JPU1Nf Q09NUElMRSIsIGdldF9jcm9zc19jb21waWxlX3ByZWZpeCgpKQogICAgICAgICAuYXJnKGZvcm1h dCEoIk9VVD17fSIsIG91dF9kaXIuZGlzcGxheSgpKSkKCg== --0000000000009e22c005df456bb6--