It's also relevant if you're writing software that's intended to be run side by side with other software.

It's relevant in the sense that you should not be using that as your guiding principle, yes.

Do any operating systems have an API for cache-only memory that the OS is free to discard at will? Maybe there should be because the other end of the spectrum is unneccessarily loading the same things over and over again even though there is lots of excess memory available. Many games are guilty of this, not really taking advantage of overspecced systems.

Actually, Serenity has this, called "purgeable memory". You map your memory in a special way, and have to re-map it to use it. If the system needs more memory, it purges those pages, making it possible to use it as a cache :^)

Neat, found this for a Linux implementation: https://nullprogram.com/blog/2019/12/29/

Most OSes will have this but it's difficult to use as a general API. The main use case is "I freed these pages and will not use them anymore, save on the bookeeping". Anything beyond that is a little dicey because if the OS does reclaim the pages you'll take a fault trying to hit the cache, which can be bad if it happens at an inopportune time. And properly signaling this to applications is hard (SIGSEGV is not a very ergonomic interface).