CopyMe: An Expression Mimicking Game to Teach Emotions to Children
Emotional development is important in the early years of growth in young children. Early intervention is especially crucial for those with psychological disorders such as autism spectrum disorders (ASDs). However, traditional paper-based interventions for emotional development are mostly laborious and difficult to employ for carers and parents, whilst current computer-aided interventions feel too much like obvious assistive tools and lack timely feedback to inform and aid progress. We hence designed and performed a pilot study on CopyMe, a touch-based tablet application that uses game design concepts and state-of-the-art computer vision techniques to provide an engaging experience as well as real-time feedback for children to learn facial expressions. In our pilot study, we found that the children, especially the ones affected by ASDs, were able to perform well in the game and generally expressed much enjoyment during play.
The core gameplay of CopyMe involves mimicking facial expressions from photographs. It is currently implemented on an Apple iPad and uses the device camera as the primary player input using facial expressions. The design was based on guidelines from prior research on games with similar goals. Throughout the game, the child will be able to see visual splines superimposed onto his/her own face on the screen whilst he/she tries to mimic the example expression. This way, thechild will be able to continuously obtain feedback on how he/she is performing by him/herself. This also enables the option of recording gameplay data to fulfill a carer's need to assess a child's behavioral patterns.
The facial expression recognition aspect of the game was a clear novelty for all the participants, demonstrated through immediate positive emotional responses when they first accessed the game level and noticed the reactivity of the application to their faces. All six participants responded positively when asked if they thought the game would be useful to teach emotions. The beneficiaries of the impact were hence primarily children from the childcare centre. It was also evident that the children with ASDs had a more enriching experience than the rest of the participants.
The CopyMe project underpins this case study. The broad goal of this project is to investigate whether game design concepts coupled with computer vision technology can effectively aid emotional development in children. Research in these projects primarily involved user studies to evaluate player experiences across multiple dimensions. These include player engagement, enjoyment and perceived usefulness of the serious component of the games on the target audience.
The research outcomes directly relate to the impact presented here.
• C. T. Tan, N. Harrold, and D. Rosser, “Designing CopyMe : an expression mimicking serious game,” in Proceedings of SIGGRAPH Asia 2013 MGIA (Presentation), 2013.
• C. T. Tan, N. Harrold, and D. Rosser, “Can you CopyMe ?: an expression mimicking serious game,” in Proceedings of SIGGRAPH ASIA 2013 MGIA (Demo), 2013.
• N. Harrold, C. T. Tan, and D. Rosser, “Towards an Expression Recognition Game to Assist the Emotional Development of Children with Autism Spectrum Disorders,” in Proceedings of WASA, SIGGRAPH ASIA, 2012.