Response: Theory-oriented evaluation for gaming and simulation

November 27, 2012

Theory-oriented evaluation for the design of and research in gaming and simulation by Willy Christian Kriz and Jan Ulrich Hense critically analyzes approaches to designing gaming simulations and how through the micro design process of such simulations, greater systems and infrastructures can be influenced on a macro level. Kriz and Hense suggest the concept of theory-oriented evaluation as a means to look at the design of simulation games beyond the “black-box model” – by testing if the simulation works, but also how and why it works.

The authors define two scientific schools of thought – the science of analysis and science of design - and argue that while both are based on different criteria and as a result are often considered separate practices, a combination of the two can contribute to the overall usability and application of a given product. In regards to game simulations, the science of design looks at using real world knowledge as means to accurately replicate the desired functionality with a focus on usability, while the science of analysis is concerned with improving knowledge of a particular domain through simulations. By integrating the two approaches, the design of a simulation becomes a process stimulating knowledge growth in its associated domain as well as maintaining a high-quality user experience.

The idea of improving education is a topic I’ve discussed in previous response entries, and Kriz and Hense raise education as one possible domain that could be improved through a “design-in-small,” “design-in-large” hybrid approach. For instance, I envision a panel of teaching practitioners and professionals pinpointing a specific issue with the present system they would like to see addressed (such as the oversight of varied student learning styles). Through focus groups of such professionals working alongside designers and developers, they might conceive of a simulation that teaches 18th century history. Perhaps in the design process they pinpoint that different students learn in different ways, and through the course of development and testing, find solutions to cause automated adjustment in the simulation based on a student’s performance. Finally, when the simulation is complete and distributed in classrooms, present flaws in the education system are undermined as students have a shared learning experience through varying degrees of  lesson, ultimately opening the door to similar ventures in the future.

This is just one example I can see Kriz and Hense’s approach being used for, though their model applies to a wide range of scientific and design-based research, perhaps even outside the realm of game simulation. The applications for this type of design approach are promising and endless.