From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-1.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.3 Received: by atuin.qyliss.net (Postfix, from userid 496) id B277C178B5; Mon, 6 Jul 2020 00:03:30 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 4F98A1777D; Mon, 6 Jul 2020 00:03:10 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id B00F21776C; Mon, 6 Jul 2020 00:03:06 +0000 (UTC) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by atuin.qyliss.net (Postfix) with ESMTPS id C552F1776A; Mon, 6 Jul 2020 00:03:02 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 8A0165C00D5; Sun, 5 Jul 2020 20:03:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Sun, 05 Jul 2020 20:03:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h= from:to:subject:date:message-id:mime-version:content-type; s= fm3; bh=NU9ndzL2bj3j0mZ09fo8ku4oFze6wbsH45Xb5YEEGQU=; b=Rfms+kQy OB67c7A597UwAFx7vCrePTSZ3bTpMOstj0SIUGXOZOzJQOsnx40OlDvHNenqpuA6 lZslDAkVuwyAQINi6YPf7QFcCjroAKOYbg3OBZLD6pIeZfFOZ3S4Vr7gb00RD0Jv MmHiuCUA3YLqGzkZf5adMeeqpevrV1W9qi1CuXwWipl8LKo+QgT/ZoGAXWEVyhwD SNjaHgQrAntZDTnZCaX70qXaJhV1x6Gw0bPd+pKuQWK7XqF6aapwhH2skcbc82/7 Vac9z0H3u5QkcTphXUaBNWWDC2IC9b2C0MmSjwPiBc9UxXxtTP0W3xNHuFpLJMA8 SwYcT0NqPuy5xg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=NU9ndzL2bj3j0mZ09fo8ku4oFze6w bsH45Xb5YEEGQU=; b=jS9neojM7xiSMPai8CSaUZMCijqNl2WIFucFcy5NF/sQH tFrOZJc+3/o+pxB6c8/+6ZC7g6Rvj8UnKZojC08kQjhTPdDXssqDNxx09KmJ3Vuj wqAh8lcEvoVr69hc+Xf2e4+bPOrs4tPmT+Od/jlipg1e/KbxnMgztccAf1SsgGZy rUtyOqOo92giZfEHvjF2CM7BGpWpdy8nG9UJ3sEUYsnNw+onL4b9QjPykFO6gFn0 et6w9qGdeap8U4P/InC8bTGWpAK7rLH9fxnVCCrAf7u4BllytfGV+P1TIc/uTaUQ VFIsdlu5Seyi91+/bLBj29bfmHmShh3ruchDM62zw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddvgddvjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkfggtgesghdtreertddttd enucfhrhhomheptehlhihsshgrucftohhsshcuoehhihesrghlhihsshgrrdhisheqnecu ggftrfgrthhtvghrnhepleeuhfeitdeujeffuefgieekteeihedthedvteefffehffefve duleejieeigefgnecuffhomhgrihhnpehsphgvtghtrhhumhdqohhsrdhorhhgnecukfhp peejledrvdefhedruddviedrudejleenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehhihesrghlhihsshgrrdhish X-ME-Proxy: Received: from x220.qyliss.net (p4feb7eb3.dip0.t-ipconnect.de [79.235.126.179]) by mail.messagingengine.com (Postfix) with ESMTPA id D053D306AAA9; Sun, 5 Jul 2020 20:03:00 -0400 (EDT) Received: by x220.qyliss.net (Postfix, from userid 1000) id 770DA259; Mon, 6 Jul 2020 00:02:59 +0000 (UTC) From: Alyssa Ross To: discuss@spectrum-os.org, devel@spectrum-os.org Subject: This Week in Spectrum, 2020-W27 Date: Mon, 06 Jul 2020 00:02:57 +0000 Message-ID: <8736654ij2.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Message-ID-Hash: 7UEZD4KF3XZE74S4F7BR2KZL5ZWHTWQ3 X-Message-ID-Hash: 7UEZD4KF3XZE74S4F7BR2KZL5ZWHTWQ3 X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.1 Precedence: list List-Id: General high-level discussion about Spectrum Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain I really didn't want this to be another week where I posted about how I was still trying to patch Wayland to do virtio_wl, and I am delighted to have just discovered it's not going to be! crosvm ------ I realised that emulating accept(2) for the Wayland compositor socket in the way I'd planned would require some crosvm rework. I want to have a host proxy program that accepts the connection, then connects the connection socket to crosvm. I had made it possible to dynamically add sockets to the crosvm Wl device through the control socket, but this turned out not be enough, because crosvm would store virtio_wl sockets in a BTreeMap, and then use connect(2) to connect to the socket when asked to by the guest kernel. This works fine for e.g. connecting to a host Wayland compositor, which is what crosvm was designed for, but it wouldn't work for opening a connection socket from accept(2), because you can only connect to a listening socket. So instead, I modified the `crosvm wl add' command to take a file descriptor pointing to the connection socket. I made crosvm store sockets as an enum that looks like this: enum WaylandSocket { Listening(PathBuf), NonListening(UnixStream), } This way, when it gets asked by the VM to connect to a socket, it can either connect to a listening socket at its path using connect(2), or just use the existing file descriptor if it's a non-listening socket. A NonListening socket will be consumed by a connection, so when the VM close(2)s it, it'll go away, and on the host side the connection will finish as expected. Listening sockets can be connected to repeatedly, as before. I also added support to `crosvm add wl' for dynamic socket names. So it's possible to do `crosvm add wl wl-conn-%d', and connections will be added with names like `wl-conn-0', `wl-conn-1', etc. So it's easy to get unique names for connection sockets. The chosen name is printed by the command, so the caller knows what name to tell the VM to connect to. I also found and fixed a bug with the previous crosvm deadlock fix[1]. I had assumed that device_sock.recv(&mut []) would drop a message from the (SOCK_SEQPACKET) socket, without having to read any of it. But UnixSeqpacket::recv calls libc::read, and read(2) tells us that: > In the absence of any errors, or if read() does not check for errors, > a read() with a count of 0 returns zero and has no other effects. So this was in fact doing nothing at all. I don't know why crosvm's UnixSeqpacket::recv calls read() instead of recv(), but it's always been like that and I'm guessing this sort of thing (from recv(2)) might have something to do with it: > The only difference between recv() and read(2) is the presence of > flags. With a zero flags argument, recv() is generally equivalent to > read(2) (but see NOTES). So probably read() just looked like a nicer way to recv() when no flags were needed. But, unfortunately, zero-byte reads are when the aforementioned NOTES section becomes relevant: > If a zero-length datagram is pending, read(2) and recv() with a flags > argument of zero provide different behavior. In this circumstance, > read(2) has no effect (the datagram remains pending), while recv() > consumes the pending datagram. So, my assumption that UnixSeqpacket::recv(&mut []) would consume a message turned out to be quite reasonable -- the surprising thing was that a method called `recv' would call read() rather than recv(). I think the best fix here will be to just make it call recv() instead, rather than modifying my code to do UnixSeqpacket::recv(&mut [0]) or something, to prevent further nasty surprises with this in future. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20200614114344.22642-1-hi@alyssa.is/T/#t Wayland ------- I created API-compatible implementations of the libc sendmsg(2) and recvmsg(2) functions for virtio_wl sockets. This was quite an achievement, because the API (which allows you to send and receive data and file descriptors, as well as other things I don't intend to support) is rather arcane (see the example in cmsg(3) if you're not familiar with them). I wrote unit tests for them, and it took a long time before they worked reliably. Once I had these, though, I could find the places where Wayland called sendmsg() and recvmsg() and fall back to the virtio_wl-based implementations if the standard functions failed with ENOTSOCK. I stubbed out some stuff that isn't going to work over virtio_wl, like looking up the pid of the Wayland client through getsockopt(2). I also had to resort to a few hacks, like faking support for MSG_DONTWAIT by using fcntl(2) to set O_NONBLOCK on the socket, recv()ing from it, and then removing O_NONBLOCK again, or faking mremap(2) by munmap()-ing and mmap()-ing. We will want to clean these up later by implementing the required missing functionality in the virtio_wl kernel module. In the first case, at least, this should be pretty straightforward, because it supports non-blocking operations if the socket is O_NONBLOCK -- it just needs to accept a MSG_DONTWAIT option as well. The VIRTWL_IOCTL_{SEND,RECV} syscalls don't currently have a flags argument, so that'll need to be added. I implemented this bit by bit, at every step trying to run Alacritty on my host system, connected to the virtio_wl Wayland server socket through the accept() proxy, and using strace and some printf()-debugging to see where the Wayland compositor in the VM would get stuck, and about an hour ago, it finally worked! For the first time, a Wayland compositor running in a VM can display an application running outside of it. (Obviously we'll want the application to be running in another VM rather than on the host, but that's similar enough that it probably works already -- I just haven't tested it yet.) This feels like a huge achievement. I've been working towards it for so long. Next week, I'll be cleaning up this code and posting patches for all of it. Then I'll probably move on to other sorts of device virtualization, like running a virtual network device in a VM. I'm feeling so much more positive about the direction of the project than I was before. It's been difficult to make myself keep going making little progress for the last couple of weeks, and it's great that I've managed to pick things up again so much. I hope that the level of detail in this email is enough to make up for the brevity of last week's! I'm sending late again, too, but only by a couple of minutes -- I didn't expect this email to take over an hour to write, but there we go. Thanks for reading! I hope you're looking forward to seeing where things go from here as much as I am. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEH9wgcxqlHM/ARR3h+dvtSFmyccAFAl8CajEACgkQ+dvtSFmy ccC4aA/8C1wZz24RkqKLiIkDLMLf+D2jylAgQOAnG97dKiU9T19J7BPIdXdLDAjq Rx43sOoGZlHn2IFAMFi8iAmU2i3jlA3xLO0i1lXzNmkxP6+dNIo5TccgG0LoTMFL ci9iEk+z02/s1YO/s/rYLMWE9CO/gXDz/i/Dsm2G2Tau/6QOPdaj7zeQ8AwN9S3s lYimHc7CP4g2GCKyL9cXOg+jRwL+ZzIXAaQXMD6c056wh7Lf8tURMlJwjr3uq3Dr Oute2yhgFUxzEve1jtzkJxTBS2sgYAreYn0wM2aRJbANRainuUnHb2TtFOpdj+fQ EMBfKZiN/sgYdaDdQlZzGFYam+yOVqCK4NpFzJg2Z6yWQIFRohlFxND3KYriLBuM RYgXvuhwTY6bed+bhUmbsGy+aWEmU1nengnrqFfpGlilJlvi0H9gp9yKQ1SrWvU+ FRGRgMRwudeuwjHbljr2RGpIQxPGiK+Sxn6WBipp7YHcz151aUqVbSTAa1tZRQfv t/TvOSD8cFG7bHX04u/VMpE03SlydKE2BT84Vw3OIPPvOd50tZsm+/dINaKXwQPF 5uJQLOc50MfIjNUDYsxrWFlKhxQAFzcDClXA9LfBPiAU19l9nq4HIKwcVnrpOwOT 5IRwTUS5HCearjshahc6oPK+/kwSVntBZOnkzmH98UK2XMLjs4M= =jyhh -----END PGP SIGNATURE----- --=-=-=--