From: Thomas Leonard <talex5@gmail.com>
To: Alyssa Ross <hi@alyssa.is>
Cc: discuss@spectrum-os.org
Subject: Re: Using virtio-gpu instead of virtwl
Date: Wed, 6 Apr 2022 12:19:14 +0000 [thread overview]
Message-ID: <CAG4opy8LjkP+uB7WME_DLjKrbbFbvY=uffBtN1FE3nm66YBYNQ@mail.gmail.com> (raw)
In-Reply-To: <CAG4opy-FYHptvg+UQGBTEih8KbfaTDEZ6CGH4Rpso+gjLU2eeQ@mail.gmail.com>
On Tue, 22 Mar 2022 at 20:05, Thomas Leonard <talex5@gmail.com> wrote:
>
> On Tue, 22 Mar 2022 at 11:16, Alyssa Ross <hi@alyssa.is> wrote:
> >
> > On Tue, Mar 22, 2022 at 11:08:15AM +0000, Thomas Leonard wrote:
> > > On Mon, 21 Mar 2022 at 16:05, Alyssa Ross <hi@alyssa.is> wrote:
> > > >
> > > > On Mon, Mar 21, 2022 at 12:10:43PM +0000, Thomas Leonard wrote:
> > > > > Looking at the Linux virtio_gpu driver, it seems that using contexts
> > > > > requires virgl:
> > > > >
> > > > > static int virtio_gpu_context_init_ioctl(struct drm_device *dev, void
> > > > > *data, struct drm_file *file)
> > > > > {
> > > > > ...
> > > > > if (!vgdev->has_context_init || !vgdev->has_virgl_3d)
> > > > > return -EINVAL;
> > > > >
> > > > > https://github.com/torvalds/linux/blob/f443e374ae131c168a065ea1748feac6b2e76613/drivers/gpu/drm/virtio/virtgpu_ioctl.c#L732
> > > > >
> > > > > I think perhaps that crosvm is compiled without the "virgl_renderer"
> > > > > feature (it's not in the default set), and this is causing it to crash
> > > > > because that's also "self.default_component". I don't know how to
> > > > > compile crosvm with virgl enabled, though.
> > > >
> > > > It wasn't easy, but I got it to build[1]. I hope that helps. It adds
> > > > both virgl_renderer and virgl_renderer_next. I think virgl_renderer
> > > > is on by default with --gpu, and virgl_renderer_next is used with the
> > > > --gpu-render-server argument. Hopefully at least one of those does the
> > > > right thing — let me know!
> > > >
> > > > [1]: https://github.com/NixOS/nixpkgs/pull/165128
> > >
> > > Thanks, that is very helpful!
> > >
> > > I gave it a try, and it got a little further. But now, doing `modprobe
> > > virtio_gpu` in the VM crashes crosvm with:
> > >
> > > Stack trace of thread 2:
> > > #0 0x00007fa5fd0915f6 abort (libc.so.6 + 0x265f6)
> > > #1 0x00007fa5fcfc6bfd get_dlopen_handle.part.0 (libepoxy.so.0 + 0xc7bfd)
> > > #2 0x00007fa5fcfc7366 epoxy_egl_dlsym (libepoxy.so.0 + 0xc8366)
> [...]
> > >
> > > It looks like it should be printing a message to stderr before calling
> > > abort, but I don't see it
> > > (https://github.com/anholt/libepoxy/blob/1.5.9/src/dispatch_common.c#L315).
> >
> > Did you try --disable-sandbox, like I suggested in my other mail?
> > The sandbox blocks writing error messages, and is something I frequently
> > trip over when trying to use crosvm.
>
> It's not very easy because --disable-sandbox seems to conflict with
> --shared-dir, which I use for lots of things.
I got around this by changing `create_gpu_device` to use `let jail =
None;`, so only the GPU device isn't jailed.
I suspect the minijail config needs updating for NixOS (e.g.
https://github.com/google/crosvm/blob/main/src/linux/gpu.rs#L82).
I tried, but failed, to figure out the protocol. I did manage to get a
test application showing a little animation, but it crashes after a
few seconds.
The basic idea seems to be:
1. You allocate a page of memory shared with the crosvm on the host.
2. You tell crosvm to read messages from the host compositor and write
them to this page.
3. After doing this, crosvm signals the guest, which reads the data.
The shared page is referred to as a "ring", but it's not used as a
ring buffer. The host always writes to the start of it.
Separately, to allocate an image buffer:
1. You tell crosvm the width and height, etc.
2. It assigns a blob_id and writes it to the shared page.
3. You wait for the operation to complete, then use the blob_id to
create the buffer.
The problem is that both these operations write to the same page, and
they race! So sometimes the image information overwrites the Wayland
data, or the Wayland data overwrites the image information, and then
it crashes.
This image_query function shows the problem:
https://chromium.googlesource.com/chromiumos/platform2/+/refs/heads/main/vm_tools/sommelier/virtualization/virtgpu_channel.cc#499
It asks for the image information to be written to "ring_addr_" and
then reads it from there. But at the moment when the function is
called, ring_addr_ may contain Wayland protocol data that hasn't been
read yet. I didn't test it with Sommelier, but that's the problem I
had in my code and I don't see how Sommelier's code can work in
general. Sommelier caches the results, so it might not hit this case
too often. I didn't use a cache, and also added a small sleep to my
code to make the problem easier to reproduce.
Anyone have any ideas how this is supposed to work?
--
talex5 (GitHub/Twitter)
http://roscidus.com/blog/
next prev parent reply other threads:[~2022-04-06 11:19 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-05 19:27 New user getting started questions Thomas Leonard
2021-01-05 20:09 ` Michael Raskin
2021-01-06 7:04 ` Alyssa's break Alyssa Ross
2021-01-06 9:11 ` Michał "rysiek" Woźniak
2021-01-06 7:00 ` New user getting started questions Alyssa Ross
2021-01-06 15:56 ` Thomas Leonard
2021-01-07 11:38 ` Thomas Leonard
2021-01-07 15:33 ` Thomas Leonard
2021-01-14 12:29 ` Alyssa Ross
2021-01-14 12:51 ` Alyssa Ross
2021-01-20 13:04 ` Thomas Leonard
2021-01-27 17:31 ` Thomas Leonard
2021-03-07 12:52 ` Thomas Leonard
2021-03-09 16:59 ` Qubes-lite With KVM and Wayland Alyssa Ross
2021-03-10 14:19 ` Thomas Leonard
2021-03-10 22:34 ` Alyssa Ross
2021-03-09 16:25 ` New user getting started questions Alyssa Ross
2021-03-13 7:21 ` Thomas Leonard
2021-03-13 13:52 ` Alyssa Ross
2021-10-30 12:58 ` Thomas Leonard
2021-11-03 11:36 ` Alyssa Ross
2021-11-03 18:27 ` Thomas Leonard
2021-11-10 12:58 ` Alyssa Ross
2021-11-10 12:00 ` Thomas Leonard
2021-11-11 11:09 ` Alyssa Ross
2021-11-11 16:12 ` Thomas Leonard
2021-11-12 10:47 ` Alyssa Ross
2022-03-13 15:08 ` Thomas Leonard
2022-03-15 14:06 ` Alyssa Ross
2022-03-15 20:23 ` Alyssa Ross
2022-03-16 16:18 ` Using virtio-gpu instead of virtwl Thomas Leonard
2022-03-16 16:54 ` Alyssa Ross
2022-03-21 12:10 ` Thomas Leonard
2022-03-21 16:05 ` Alyssa Ross
2022-03-22 11:08 ` Thomas Leonard
2022-03-22 11:16 ` Alyssa Ross
2022-03-22 20:05 ` Thomas Leonard
2022-04-06 12:19 ` Thomas Leonard [this message]
2022-04-13 17:12 ` Thomas Leonard
2022-04-14 13:57 ` Alyssa Ross
2022-04-19 12:58 ` Thomas Leonard
2022-04-19 12:01 ` Alyssa Ross
2022-05-15 15:20 ` Thomas Leonard
2022-05-16 11:55 ` Alyssa Ross
2022-05-18 9:55 ` Thomas Leonard
2022-06-05 16:29 ` Thomas Leonard
2022-08-09 12:00 ` Alyssa Ross
2022-10-10 15:16 ` Thomas Leonard
2022-10-10 16:53 ` Alyssa Ross
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAG4opy8LjkP+uB7WME_DLjKrbbFbvY=uffBtN1FE3nm66YBYNQ@mail.gmail.com' \
--to=talex5@gmail.com \
--cc=discuss@spectrum-os.org \
--cc=hi@alyssa.is \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).