Standing on the Shoulders of Giants

Sir Godfrey Kneller, Sir Isaac Newton, 1689justine-cassell

To understand this blog, it’s best to begin with an explanation of the work of Dr. Justine Cassell, principal investigator of the ArticuLab at Carnegie Mellon University. She has made significant progress in the field of embodied conversational agents, which are computer programs that are displayed with a body image and which try to imitate human behavior in some way (verbally or nonverbally or both) in order to help humans; for example, by teaching them math or being their personal assistant (like Apple’s Siri). The ArticuLab, where I do all research posted on this blog, studies human interaction in sociocultural contexts and designs computational systems that are sensitive to these contexts. The RAPT project (Rapport Aligning Peer Tutor) in the ArticuLab is headed by Michael Madaio. RAPT is researching optimal strategies for teaching math to middle school aged children. Recent research in learning sciences has shown that students learn better when they form a friendly bond with a tutor, and also that virtual tutors can improve student performance by up to two standard deviations. RAPT capitalizes on these two important discoveries, by building a virtual tutor that can respond in real-time to human friendship-building behavior (also called rapport). Hopefully some day, such a virtual tutor will make human learning not only more efficient, but fun and engaging.

This is where my personal work at the ArticuLab begins (summary can be found at the end of this post). By standing on the shoulders of giants, we can make important discoveries without reinventing the wheel. To that effect, my first week began with a literature review. My goal for this first week is to use the literature review to find research questions for future investigation in one of three categories (as described in the project proposal submitted to CREU):

A. How does a student’s learner profile (e.g. degree of extroversion or conscientiousness)      and the design of a virtual tutor (e.g. its ability to manage face) affect the building of          rapport between virtual tutor and tutee?

B. How does rapport impact learning (behaviors and outcomes) differently between                human-human interactions and human-virtual agent interactions?

C. Does rapport that is built during the tutoring process cause greater student                          engagement in the tutoring process and how can we measure that engagement (e.g. by      counting explanations or utterances)?

In approximately 9 hours, I explored 11 articles. I began by first reading articles authored by ArticuLab members to find out what the ArticuLab’s response to categories A through C are. I specifically sought articles loosely covering the subject of categories A through C and discarded the other articles. Next, I read the articles cited in the aforementioned articles. Finally, I picked the following broad questions for research, out of the countless fascinating avenues of research in this burgeoning field:

  1. Are certain personalities more likely to insult or disengage from virtual agents?
  2. Are there gender differences in problem solving behavior and to what extent are these important?
  3. How does anxiety predetermine what rapport building strategies a student uses (which a virtual tutor must be sensitive to)?

Below is a review of sources that led me to these questions (1-3), with numbers corresponding to the questions (1-3) above. The complete citations for the works mentioned below can be found in the bibliography at the end of this post.

  1. Question 1 fits into Category A because it seeks a connection between innate characteristics of the student (e.g. personality traits) and rapport-destructive behaviors (e.g. insults, abuse). Noftle and Shaver (2006) described how a human’s personality affects the kinds of relationships which that human seeks with other humans; for example, highly anxious people tend to fear attachment (high rapport) with others. Furthermore, Park et al. (2014) demonstrated that anger, frustration, and narcissism are all significant (p < 0.01) positive predictors of rude, rapport-destructive behavior in humans. Due to the persona effect (Johnson et al., 2004), whereby humans treat intelligent virtual agents as social actors, one can assume the observations on human-human relationships carry over to human-virtual agent relationships. Therefore, the personalities of the students being tutored can destroy rapport with virtual agents, namely traits like neuroticism or narcissism. These traits are part of the Big Five Personality Traits, and we measured these traits through surveys in prior experiments with human-virtual agent dyads. Putting the articles together, Question 1 arises. Due to the fact we have the data to answer this question, it seems like a promising direction for further research.
  2. Question 2 is derived from the research of leaders in linguistics research using feminist ethnography to rebut old theories of learning based on heterosexist ideologies, which polarize boys’ behavior as solely conflictive and girls’ behavior as solely cooperative. The question of gender differences appears in this blog due to Category B; I am interested in the effects that differences in gender have on communication styles and how this could cause a virtual agent that is insensitive to these differences to have more difficulty building rapport because it appears so inhuman. In a book chapter written by Norma Mendoza-Denton (1999), she describes the polysemy (roughly, multiple meanings) of the word “no” in Spanish spoken by Latina high school students. Sometimes it’s used as an affirmative word and sometimes as a prelude to a summary of what was said by previous speaker; both uses are not the conventional use of “no” for confrontation or negation. Additionally, Ardington (2006) describes how, in a tutoring paradigm, the speech pattern of two high school girls oscillates between teasing wordplay and directions to focus. For example, when completing homework, one girl makes a face threatening statement via text in a joking manner (“dingbat!”, which is a cute marsupial) and the other girl demonstrates she is not offended by the joke by replying “dingbat!” which is followed by “stop it! we’re supposed to be doing homework!”. Ardington (2006) claims this kind of play-talk often goes on in the background of serious activity. Thus, Question 2 arises because it would be beneficial to rapport-building efforts to have a virtual tutor that engages in play-talk, not necessarily only with girls but also with boys and any gender; this makes the agent appear less artificial or stoic.
  3. Question 3 fits into Category A. Question 3 asks whether anxiety, as measured by our survey, can affect rapport building behavior. Kang et al. (2008) compared personalities of different subjects to their feelings of rapport with a virtual agent that used nonverbal cues (e.g. posture mirroring, head nods). Kang et al. found that each personality corresponded to the pursuit of one of three components or rapport: coordination, mutual attentiveness, or positivity (giving each other happy emotions). For example, highly agreeable people correlated highly with self-rated reports of coordination with the virtual agent, whom they were giving instructions to but were told it was human. Overall, agreeableness correlated more significantly with rapport than any other trait in the Big Five Personality group. This leads to the question of which personality traits do and don’t influence rapport. More interestingly, Question 3 asks what the effect of anxiety is, because Kang et al. found conflicting results that higher anxiety people do not benefit any more than low anxiety people from building rapport with an agent (whom they think is human). These questions both fit into Category A because the learner profile in Category A is derived from our surveys that measured the Big Five Personality traits in all our subjects.

Once a week, I will update this blog with challenges I faced in my research. surprising challenges posed by Michael’s rejection of a gender difference question. This week was not too tough but I faced one unexpected challenge (mentioned later). Luckily, I’ve already done research on learning sciences, so my research for this literature review did not have to be so extensive. I knew what topics are best suited for our lab, and what kinds of rabbit holes to avoid (such as topics in learning sciences that are too broad or narrow). One big challenge was not being able to access my previous research for this lab, because the administration needed time to renew my access to its server. This precluded me from accessing my previous work, and building off of it. However, due to my experience at this lab this was not a big setback.

The surprising challenge I faced was the controversy behind my Question 2. After an hour meeting with my research mentor, where we discussed our future plans for this semester, we came to my Question 2. It became immediately apparent that in asking Question 2, we do not attempt to make a hypothesis that stereotypes a gender as being limited to one learning behavior over another. Clearly, Question 2 is controversial due to the involvement of gender. We decided that we would carefully explore the subject if it ever arises, and personally, I would like to modify the question to address speech patterns in middle school students in general, not just girls, and avoid heading in the direction of separating one gender’s behavior from another.

SUMMARY: I read 11 articles in 9 hours, created 3 research questions, and met with my research mentor for another 2 hours to get tips on research, make concept maps, and discuss progress. We agreed that we would meet this Thursday to narrow down the research questions even more, and then hopefully move on to the next stage, which is developing a hypothesis to answer the question.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s