Large or lengthy disruptions to the card payment system are threats that can cause crisis in society, especially in countries where other payment options are scarce. This paper presents a study that provides suggestions on how to improve a simulation game used to increase societal resilience to payment system disruptions. Questionnaires and interviews have been used to investigate how 16 participant in crisis exercises experience realism, relevance and validity in such exercises. Suggestions on how to improve the simulation game are provided, such as improvements to the graphical interface and introducing supporting roles from the exercise management.