Paper Reading #4 - “Oh, dear Stacy!” Social Interaction, Elaboration, and Learning with Teachable Agents

Intro:
     Title: “Oh, dear Stacy!” Social Interaction, Elaboration, and Learning with Teachable Agents
     Author Bios

1. Amy Ogan
• http://amyogan.com/
• Research: Virtual Agents
• Postdoctoral Fellow at Carnegie Mellon University in the Human-Computer Interaction Institute
2. Samantha Finkelstein
• http://www.samanthafinkelstein.com/
• Research: Human-agent interaction, collaborative learning, rapport, play, culture, educational games
• Ph.D. in Human-Computer Interaction, Carnegie Mellon University
3. Elijah Mayfield
• http://www.cs.cmu.edu/~emayfiel/
• Research:  representing a deeper understanding of natural language computationally
• Ph.D. student at Carnegie Mellon University, more specifically at the Language Technologies Institute in the School of Computer Science
4. Claudia D'Adamo
• http://www.linkedin.com/in/claudiadadamo
• Research: Human Computer Interaction, Graphic Design, Cognitive Psychology, Robotics, Machine Learning, and Data Mining
• Student at Wheaton College
5. Noboru Matsuda
• http://www.cs.cmu.edu/~mazda/
• Research:  artificial intelligence and cognitive theories of learning to educational problems
• Carnegie Mellon University
6. Justine Cassell
• http://www.justinecassell.com/
• Research: Deconstructing human conversation and storytelling in such a way as to give machines conversational, soical, and narrative intelligence to allow them to communicate with people; Impact and benefits of technologies such for learning and communication
• Carnegie Mellon University

Summary:
Teachable agents are systems that allow children to tutor them. The system learns as the child explains things to it. The goal of these programs is for the kid who is the tutor to learn from teaching. The goal of this experiment was to see the affects of cognitive moves, social moves, and inside-system vs. outside-system language on learning gains.

Figure 1. Stacy Interface [1]

The study used a Learn by Teaching environment, called SimStudent. There was an avatar, named Stacy, that would be taught by a child. The appearance of the environment can be seen in Figure 1. The children showed up on the first day, took an algebra pretest and then started working with Stacy to help her past 4 tests by teaching her with practice problems. The students were instructed in a think-out-loud method that allowed the researchers to document what the students said. The students returned the second day to finish the lesson and to take an algebra post test.


 

Figure 2. Correlation between Out-side alignment and learning gain [1]

Through  evaluating the differences in scores and the different type of utterances made, the researchers were able to come to several conclusions. Contrary to many studies, this one showed a decrease in learning when there was an increase in elaboration. It seems this was because when there was an increase in elaboration, there was also an increase in outside-aligned speech (she, it). This study showed that there was a strong correlation between outside-alignment and a decrease in learning gain. They also found that outside-alignment was often caused when Stacy got things wrong. This doesn't occur with peer-peer teaching, so they suggested observing the social activities that occur in these situations and implementing that in the program. They also found that when students treated Stacy like a peer and not a tutee or computer and used negative social moves like teasing, they had the highest learning gains. [1]

Related work not referenced in the paper:

1. Teachable Agents: Combining Insights from Learning Theory and Computer Science

• http://www.teachableagents.com/papers/pre2001/aied99.pdf
• This paper mainly discusses the framework of Teachable Agents, which my paper never did.

2. Pedagogical Agents for Learning by Teaching: Teachable Agents

• http://www.teachableagents.com/papers/2006/Final-edtechTA.pdf
• This paper discusses the features of TAs and new places to use them, in contrast to my paper where they use them to prove another idea

 3. Intelligent user interface design for teachable agent systems

• http://dl.acm.org/citation.cfm?id=604054
• This paper evaluates the performance of TAs and their interface. The paper I had didn't evaluate the actual agent, so much as how people related to the TA.

4. Extending Intelligent Learning Environments with Teachable Agents to Enhance Learning

• http://teachableagents.org/papers/2001/aied2001.pdf
• This paper looks into how to make a TA and evaluating how effective it is.

5. Interactive Metacognition: Monitoring and Regulating a Teachable Agent

• http://www.public.asu.edu/~kvanlehn/ITScourse2009/Readings/Schwartz%20Chase%20Chin%20Oppezzo%20et%20al%20in%20press.pdf
• This paper proposes that TAs allow students to develop content knowledge and metacognitive skills

6. Learning by Guiding a Teachable Agent to Play an Educational Game

• http://celstec.org/system/files/file/conference_proceedings/aeid2009/papers/paper_133.pdf
• This paper talks about using a TA for an educational game and about how this shows promise as a way to help low ability students

7. Teachable Agents Learning by Teaching Environments for Science Domains

• http://www.aaai.org/Papers/IAAI/2003/IAAI03-015.pdf
• This paper discusses the Betty's Brain experiment in a Nashville school and how they are developing a new version that focuses on formative assessment and the teaching of self regulated strategies

8.  Measuring Self-Regulated Learning Skills Through Social Interactions in a Teachable Agent Environment

• http://www.apsce.net/RPTEL/2010_05_02_4.pdf
• This paper again focuses on Betty. It describes an experiment that proved that TAs improve learning and specifically learning metacognitive skills.

9. Teaching about Dynamic Processes A Teachable Agents Approach

• http://dl.acm.org/citation.cfm?id=1562561
• This paper discusses changes to Betty to teach students about a river ecosystem

10. Do Learning by Teaching Environments with Metacognitive Support Help Students Develop Better Learning Behaviors?

• http://www.teachableagents.com/papers/2007/MetacognitiveSupportBehaviors.pdf
• The paper describes an experiment that showed that students who teach a TA and receive metacognitive support learn more than just teaching the agent or the agent teaching them.

Almost every paper I saw either had to do with Betty or developing a TA. None of the paper seemed to address social interactions with the TA at all. Because of this, I feel this paper is very novel in that regard.

Evaluation:
The experiment was performed on 12 students (2 girls and 10 boys) in 7th through 10th grade. The way the experimenters tested their hypotheses was by giving students and pre and post test covering algebra. They then compared the results to the dialogue they had with the system and observed any papers. This is a very objective, quantitative (test scores and numbers of types of interactions). It was systemic because it measured the TA has a whole, and not the specific parts of the system that influenced learning. I thought this was a very good way to evaluate the hypotheses. I do thing a greater number of students should have been used though.

Discussion: 
I completely agree with the affect of teachable agents. I am a peer teacher for the department, and I feel that I learned more peer teaching classes than I did actually taking the class. I also try to stay on a social level with the students in the lab if I can. It puts both of us at ease, and I feel improves the learning environment a lot. I could definitely see teachable agents as a way to help students learn in the future. It's already helped me personally without that even being my purpose.

Reference Information:
[1] “Oh, dear Stacy!” Social Interaction, Elaboration, and Learning with Teachable Agents: http://delivery.acm.org/10.1145/2210000/2207684/p39-ogan.pdf?ip=128.194.132.227&acc=ACTIVE%20SERVICE&CFID=151349081&CFTOKEN=89389542&__acm__=1346829303_239009bf01d4a35efc3e90fb7fb2cade
[2] All papers listed were found using http://scholar.google.com/