I don't know about it being practical to map the GPU into userspace. Most systems only have the one GPU, and it being handed off to be managed by userspace (presumably via IOMMU allocation, like happens when you "pass through" a GPU to a particular virtual machine) means the kernel now can't use it. So the game can draw to the screen, but the OS can't. Which sucks for startup/shutdown, and any time things go wrong.
And yeah, if you imagine having to write a game as an OS kernel driver, then yeah, that's probably impractical. You don't want to have to program a game using kernel APIs.
But imagine instead, taking a regular OS with protected memory, and:
• Stripping out the kernel logic during context switches / interrupts, that de-elevates userland processes down to "ring 3" (or whatever the equivalent is on other ISAs.)
• Ensuring the kernel is mapped into every process's address space at a known position.
• Developing a libc where all the functions that make syscalls, have been replaced with raw calls to the mapped OS kernel functions that those syscalls would normally end up calling.
So you're still developing "userland" applications (i.e. there are still separate processes that each have their own virtual address space); but there's no syscall overhead. So a series of synchronous kernel syscalls to e.g. allocate O(N) tiny video-memory buffers, would be just as fast as (or faster than) what mechanisms like io_uring enable on Linux.
