Queen Mary PhD student contributed to the new generative AI model in gameplay creativity

Queen Mary PhD student Marko Tot

A groundbreaking study published today in Nature presents a significant advancement in generative AI for the creative industries, particularly in video game development. The research, primarily led by Microsoft Research, explores how AI can support and enhance human creativity by aligning with the iterative and exploratory nature of game ideation. 

The paper introduces the World and Human Action Model (WHAM), a state-of-the-art generative AI model trained on human gameplay data. WHAM demonstrates three critical capabilities that support creative ideation: consistency, ensuring generated content adheres to logical game mechanics and narratives; diversity, allowing for novel gameplay sequences; and persistency, maintaining user modifications over iterations. 

The study builds on insights from a user study involving 27 professionals across the gaming industry. The research underscores how game developers require AI to support divergent thinking, where multiple creative possibilities are explored, and iterative practice, where content is refined through successive modifications. WHAM addresses these needs by generating diverse yet coherent gameplay sequences and allowing designers to guide AI outputs more effectively. 

The research has implications beyond gaming, paving the way for broader applications in creative AI, from interactive storytelling to content generation in film and music. 

One of the co-authors of the study, Marko Tot, contributed to this research while completing an internship at Microsoft Research as part of his PhD at Queen Mary University of London. Reflecting on the findings, Tot remarked:  

“Working alongside Microsoft Research to develop WHAM was an incredible experience. It was a pleasure to work with the Game Intelligence team throughout the summer and to contribute to such a novel project. Having a tool that can instantly shape your ideas into an interactive game world opens up many new possibilities for game design. Testing various new ideas and making them persist in the virtual worlds with WHAM allows for rapid iteration and improves the creative flow of designers. Seeing characters come to life and seamlessly integrate into the game environment with just a button click makes the creative process more intuitive and accessible. I’m excited to see how it inspires the next generation of game creators.

"While working on WHAM was one of the outcomes, the real thing I got out of the internship were friendships that outlasted my four months there during summer. Working on cutting edge research while also connecting on a personal level makes you feel like a more than just a temporary intern."

In an effort to encourage further innovation, Microsoft Research has released the WHAM model weights, an evaluation dataset, and the WHAM Demonstrator tool for public exploration and research. These resources are now available at https://huggingface.co/microsoft/wham. 

As generative AI continues to evolve, WHAM sets a new benchmark for how AI can be integrated into creative workflows, marking an important step towards human-AI collaboration in ideation and design.