Paper Reading #6 - Making Calibration Tasks Enjoyable by Adding Motivating Game Elements

Intro:
     Title: Calibration Games: Making Calibration Tasks Enjoyable by Adding Motivating Game Elements
     Author Bios

1. David R. Flatla
• University of Saskatchewan, Saskatoon, Saskatchewan, Canada
• http://hci.usask.ca/people/view.php?id=13
2. Carl Gutwin
• University of Saskatchewan, Saskatoon, Saskatchewan, Canada
• http://www.cs.usask.ca/~gutwin/ 
3. Lennart E. Nack \
• University of Ontario Institute of Technology, Oshawa, Ontario, Canada 
• Faculty of Business and Information Technology
4. Scott Bateman
• University of Saskatchewan, Saskatoon, Saskatchewan, Canada
• http://hci.usask.ca/people/view.php?id=18
5. Regan L. Mandryk 
•  University of Saskatchewan, Saskatoon, Saskatchewan, Canada
•  http://hci.usask.ca/people/view.php?id=1

 Summary:
Many human-computer systems need calibration to occur for the best results. Calibration though usually involves repetitive and tedious tasks that can take awhile to complete. Because of this, the calibration does not occur, and the result is a less than desirable experience with the system. The authors want to make calibration more enjoyable and engaging, and therefore encourage its completion, by the creation of calibration games.

There are two types of calibration. The first addresses human limitations, and the second deals with the computer system measurements. There are many forms of calibration under these two types, but all of them have a core task that is used to measure the calibration. As long as this core task is maintained within the game, the game should perform the calibration as well as the standard calibration form. The researches first analyzed the different calibration forms to identify what the core task was for each and determine an appropriate game form that aligns with it. To see the different forms of calibration, the core task, and the type of game that tests the same thing, refer to figure 1.

Figure 1. The different forms of calibration (left), the core tasks (middle), and the matching game forms (right)

Each game would need to be designed with some mix of challenge, theme, reward, and progress to keep th e user interested and engaged, but they cannot interfere with the results of the calibration. They developed 3 different games to test their idea.

Figure 2. JND Game

The first game was for calibrating color just-noticeable-differences(JNDs). They developed a space invader game, where the user would shoot targets with a missile when they were able to differentiate it from the background. This would work because the core task to both the standard and game version is the same (telling the difference between colors). They used high scores and new items to reward players and encourage accuracy.

The next game they made was for calibrating C:D parameters for targeting. The standard way this occurs is through repeatedly selecting targets on a plain background. They created a game for this by identifying 'targeting' as the core task and creating a game around that. They made a shooting-gallery style game. The game had a plot (save the universe from the 'blorgs'), a timer, lasers and explosives, sound effects, and bonuses to make it more exciting and rewarding. The game basically involved them selection 'blorgs' to shoot them, and after so many hits, they receive rewards, such as gold starts and high scores.

Figure 3. BabyLaunch game

The last game they developed was for calibrating a physiological sensor, specifically a respiration chest strain sensor where to calibrate the user needs to breath deeply during the calibration trails. They made a baby launching game to increase enjoyment during this. The deeper you breath, the farther the baby will be launched. They provided rewards such as progress (current maximum distance) and colorful visuals.



They performed tests (discussed in the evaluation section) to evaluate the games. The experiments showed two things. The first was that the games were more fun than the standard calibration technique. People typically perferred to use the game. The second finding was that, while using the game method did affect the results, it should not affect the actual value of the calibrations.


Related work not referenced in the paper:

1. A flexible new technique for camera calibration   
• http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=888718&tag=1
• This describes a new way to calibrate a camera that is easier and quicker than previous methods  

2. Heuristics for designing enjoyable user interfaces: Lessons from computer games  

   • This paper discusses ways to make user interfaces more enjoyable based on lessons from game design
   • http://dl.acm.org/citation.cfm?id=80175
3.  Learning from Games: HCI Design Innovations in Entertainment Software
• http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.122.6347&rep=rep1&type=pdf
• This paper discusses things we could learn about designing UIs from video games.
4.  Games and the Design of Human‐Computer Interfaces
• http://www.tandfonline.com/doi/abs/10.1080/0954730940310208
• This paper discusses how computer games could be uses to more effectively design other computer systems
5. Applying game achievement systems to enhance user experience in a photo sharing service
• http://dl.acm.org/citation.cfm?id=1621859
• I think these two articles relate because they are using games to boost enjoyment in tasks you wouldn't expect. 
  
6. Framework for Designing and Evaluating Game Achievements
• http://www.digra.org/dl/db/11307.59151.pdf
• This paper discusses achievements, which was an important part of the games developed in this research paper 
7.  Exploring Aesthetic Ideals of Gameplay
• http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.155.4201&rep=rep1&type=pdf
• This paper  talks about the different ideals that make a game enjoyable. These type of principles were used during the making of the calibration games.
8. What does it mean to understand gameplay?
•  http://www.sea-mist.se/fou/forskinfo.nsf/all/4cdfcc11dad2f9fec12575c8003ee24d/$file/Lindley%20-%20What%20does%20it%20mean%20to%20understand%20gameplay.pdf
• I think this article is related because the researchers had to research what gameplay was before they could develop the calibration game.

9. Just blink your eyes: a head-free gaze tracking system

   • http://dl.acm.org/citation.cfm?id=765891.766088

   • This paper described a gaze tracking system that had a short and easy calibration method.

     
     

      10. Uncalibrated stereo hand-eye coordination

• http://www.sciencedirect.com/science/article/pii/026288569490071X
• This paper describes a robot that handles the calibration for the developed system.

This paper is very innovative. I had to struggle to find any papers even a little related to calibration games. All the papers I have listed have to do with small parts or related issues, but none have to do with calibration games.


Evaluation:

Table 1. Likert Statement Sample questions/results

They used 12 individuals from the college to test each game. They would do both the standard and game version of the calibration and then answer questions over the experience. They compared the calibration results to test accuracy and the individuals remarks on enjoyment. They used a 5-point Likert scale to evaluation the individuals responses. To see the questions asked in the study, refer to table 1. I think they should have asked some open-ended questions along with the Likert scale. They made a lot of assumptions based of these results that would have been better answered in a open-response type question.



Discussion: 
I thought this was a very interesting paper. It was well writing and was a simple, unthought of idea. I could find no reference to this idea in any other paper, and if it really doesn't cause issues with the results, I think it would be very successful. No one likes to do calibration, and if the situation calls for a very long calibration like the ones described in the paper, I wouldn't want to do it. Even though I know it would hurt my experience, I just couldn't take the time to sit through the boring procedure. If it was fun, like the games would be, I would be much more likely to do it. It is a very simple idea though, so I am surprised no one tried it earlier.

Reference Information:
[1]Making Calibration Tasks Enjoyable by Adding Motivating Game Elements: http://delivery.acm.org/10.1145/2050000/2047248/p403-flatla.pdf?ip=165.91.10.211&acc=ACTIVE%20SERVICE&CFID=151349081&CFTOKEN=89389542&__acm__=1347153358_edcf5091a82f0bf5a95d001ed7563fe9
[2] All papers listed were found using http://scholar.google.com/