Microsoft shows progress toward real-time AI-generated game worlds

That demonstrator currently generates the resulting video based on pre-recorded inputs, at a rate much slower than necessary for actual live gameplay. In a private demonstration for press, though, Microsoft also showed an early prototype of a real-time WHAM-powered video-generation tool, which instantly generates new frames of gameplay based on immediate inputs from the user. Users can even jump from scene to scene instantly just by feeding a fresh set of sample frames into the system.

That kind of real-time, “generate as you go” world model is something of a holy grail for this branch of AI research. And while the current version Microsoft showed off “is definitely not the same as playing the game,” as Hoffman said during the demonstration, it’s also “decidedly not like a traditional video game experience,” she said. “It has a new quality. It’s really interesting to explore and see what I can do in this setting.”

Don’t get your hopes up for a new wave of AI-generated games any time soon, though. Microsoft’s prototype WHAM tool is still severely limited to a very muddy 300×180 resolution (comparable to a screen on the original Nintendo DS) at 10 frames per second—well below the playable baseline for modern games.

And despite all the much-ballyhooed improvements in consistency and persistence, there’s still an ethereal, dreamlike quality to many of the objects shown, even in the low-res WHAM footage. The player character, in particular, tends to morph and stretch like a shapeshifter rather than a tight player model with a solid and consistent skeleton.

Still, Microsoft says it hopes WHAM is a first step toward a future where AI can craft high-end interactive experiences at the drop of a hat. “Hopefully this gives you a sense of just what we might be thinking about as we start to work towards interactive experiences that are generated on the fly by these real-time-capable generative AI models,” Hoffman said.