Microsoft Xbox Unveils an Impractical AI Model for ‘Gameplay Ideation’

Yesterday marked an intriguing milestone for Microsoft Xbox as they introduced Muse, a generative AI model crafted specifically for “gameplay ideation.” This launch was accompanied by an article on Nature.com and a blog post that included a YouTube video. If you’re scratching your head over “gameplay ideation,” Microsoft explains it involves creating “game visuals, controller actions, or both.” However, despite the flashy terminology, the practical applications are quite constrained, and it’s not revolutionizing the existing game development processes.

Despite these limitations, the data behind Muse offers some curious insights. The model was trained extensively using H100 GPUs, requiring around one million updates to extend a mere second of actual gameplay into an additional nine seconds of simulated gameplay that aligns with the engine’s accuracy. Most of the training data came from sessions of existing multiplayer games.

Instead of running this on a single computer, Microsoft employed a cluster of 100 Nvidia H100 GPUs, significantly ramping up costs and energy consumption for output that offers just about nine more seconds of gameplay at a modest resolution of 300×180 pixels.

The standout feature showcased by the Muse team was its ability to replicate existing elements and enemies within a game environment, maintaining their functions. This approach raises eyebrows—considering the immense hardware costs and power use—when developers could just use existing tools to achieve similar in-game expansions.

While it’s fascinating that Muse can maintain object permanence and mirror the original game’s actions, the broader utility remains debatable, especially when placed alongside well-established and effective traditional video game development pipelines.

Even though Muse might someday achieve more captivating feats, it currently finds itself among numerous projects aiming to replicate gameplay through AI alone. Despite some success in maintaining engine accuracy and object permanence, its inefficiency as a method to develop, test, or experience games leaves me questioning its attractiveness. After spending hours diving into the supporting material, I still struggle to understand why anyone would choose this over proven methods.