Professor Kim McDonough – Professor Pavel Trofimovich | New Insights into Non-verbal Aspects of Second Language Learning
Although most people study a second language in school, this education alone is seldom sufficient to achieve the level of proficiency needed to live in a foreign country. But in a world where global migration is increasingly common, learning a second language has become essential for millions of people. Professors Kim McDonough and Pavel Trofimovich at Concordia University, Canada, study language learning, and in doing so, are helping those trying to become competent in the use of a second language.
The Need to Communicate
Most of us will have seen commercials in which a tourist in a foreign country talks into a phone and an app translates the sentence for the locals who respond the same way. Everybody is understood and there are smiles all around. Unfortunately, the reality is not quite so simple. While technology is getting better at supporting simple conversations about the location of a bus stop or tourist attraction, there are significant limitations to its use. Sentences translated by artificial intelligence can be difficult, or even impossible to understand. Furthermore, for those living in a foreign country using a device for every interaction is simply impractical. In our increasingly mobile world, learning a second language, often to a very high level of proficiency, is a fundamental necessity for millions of people working or studying in a foreign country.
The Challenges of Learning Languages
Anyone who has spent time around someone trying to use another language will have stories about the often amusing mistakes they make. As Professors Kim McDonough and Pavel Trofimovich of Concordia University explain, common errors fall into three main categories: the use of incorrect/inappropriate words, so-called lexical errors; the incorrect use of grammar; and issues with the pronunciation of certain words, a problem linguists call phonological errors.
When such mistakes happen, teachers correct them using a process scientists refer to as recasting. For example, if an adult was to say ‘yesterday I go to a movie’, the teacher would typically correct this to: ‘you went to a movie’. Sometimes, learners will repeat the correct expression, and eventually, they will learn to use verbs correctly.
However, depending on the person’s age and circumstances, the mechanisms for corrections and the learners’ responses to recasting may be different. Extensive research indicates that social behaviours, interpersonal interactions, cultural factors, and attitudes can profoundly affect language learning by adults. The research group led by Professors McDonough and Trofimovich is particularly interested in how subtle non-verbal cues, such as eye contact, facial expressions and hand gestures, aid communication between adult language learners and those interacting with them.
If in Doubt: Gesticulate!
One of the questions the research group at Concordia University wanted to answer was whether seeing the face and/or the upper body of a person speaking English as a second language influenced how ‘understandable’ they would be perceived to be.
In these experiments, the researchers recorded speech samples of students – half of whom were Spanish and the other half of Chinese background – and asked fellow students (‘assessors’) to rate how easily the speakers in the videos could be understood, how fluent they were, and how strongly detectable their accent was.
To evaluate the effect of facial expressions and hand gestures, the scientists manipulated the recorded speech samples. A third of the assessors listened to the speakers while only seeing a stationary image of their faces. Another group listened to the same speech samples accompanied by a video showing visual clues restricted to the faces of the speakers – such as a smile, raised eyebrow or a frown. However, in the video shown to this group, the shoulders and hands of the speakers were ‘frozen’, thus eliminating the gestures conveyed by the movements of the torso or hands. Finally, a third group of assessors were shown the full video, closely replicating a real-world interaction with the speaker.
The findings were conclusive. The same speaker was perceived as more fluent, understandable, and less heavily accented when their face was moving, and this rating improved even further when upper body gestures and hand movements were also visible. Analysing the recorded images and the assessors’ ratings in more detail resulted in two unexpected findings. Speakers moving their eyebrows more often were perceived to have less pronounced foreign accents, while those who looked away from the listeners more often were judged to be easier to comprehend.
Anxiety and Non-verbal Cues
Predictably, previous research has shown that when speaking English as a second language, students affected by anxiety and other negative emotions performed poorly and became harder to understand. However, this research was carried out using a questionnaire that simply asked students to rate how nervous they were while speaking. To evaluate the role of anxiety more accurately, the scientists attached sensors to the fingers of the students using English as a second language to measure electrical conductance. Since skin conductance changes very rapidly when someone is stressed or experiences other negative emotions, so this allowed the scientists to monitor objectively how nervous the students were while discussing various topics.
Professors McDonough and Trofimovich and the team also explored whether certain facial expressions or hand gestures were specifically associated with periods of anxiety. To identify stress-associated behaviours, they analysed the skin conductance traces while simultaneously watching a video recording of the speakers to determine specifically what actions corresponded with higher levels of anxiety.
While the finding that anxiety makes speaking a second language harder is not new, what is important here is that the scientists were able to clearly identify gestures and behaviours associated with the students being nervous. For example, periods of anxiety were associated with more frequent blinking, and anxious students averted their gaze and tended to touch their faces and hair more often. In contrast, during more relaxed conversations, speakers were more likely to nod their heads. Interestingly, most students found talking about themselves more stress inducing than discussing more general topics.
Such findings have clear practical value in language education. Recognising stress-related behaviours in students would allow language teachers to helpfully intervene – by changing conversation topics, or by providing reassurance, for example – to reduce the impact of negative emotions on language learning.
Where Interlocutors Look Matters
In another recent project, Professors McDonough and Trofimovich and their team analysed how eye contact influenced the effectiveness of corrections provided by language teachers. Here, students learning English as a second language participated in discussions about complex topics. However, during these experiments, the ‘teachers’ listening to the student were also research assistants involved in the study, and specialist equipment was used to monitor the eye movements of both the students and the interacting research assistants. Furthermore, the conversations were also video-recorded from multiple angles.
Analysing these recordings, the scientists identified over 1,000 instances when corrections (recastings) were made by the research assistants. Exploring the type of errors and analysing eye contact (gaze) between the language learners and the teaching assistants allowed the scientists to draw conclusions about the importance of the observed exchanged glances.
The results indicated that mutual gaze immediately after a teacher made a correction was more likely to result in the learner repeating the correct word/expression. Further in-depth analysis also identified that learners were more likely to correct themselves when the mistake involved the use of the wrong word or expression (lexical error). In contrast, grammatical errors were less likely to be corrected, although mutual gaze as seen on the video indicated that learners registered these corrections too. The other interesting detail emerging was that such corrections were associated with brief eye contact, while longer exchanged glances suggested that the learner either did not understand what the error was or was unable to make the correction.
Non-verbal Cues of Non-understanding
Professors McDonough and Trofimovich also wanted to explore whether listeners gave visual cues indicating that they failed to understand a statement made by a speaker. In these experiments, 60 observers, acting as impartial assessors, listened to conversations and were asked to rate the likelihood that the listeners understood the information the speaker was trying to convey. Interestingly, when the observers could see the faces of the listeners, they were able to identify instances of non-understanding much more accurately than when the listener’s face was obscured. During episodes of non-understanding, the listeners appeared to be ‘frozen’ with changes to their posture, facial expression or the direction of their gaze being suspended – a phenomenon scientists call ‘hold’ – while waiting for clarification. The scientists first noticed that these holds were associated with periods of non-understanding in a small study in 2019 and were able to confirm this in experiments conducted with larger groups of participants in 2021 and 2022.
The work of Professors McDonough and Trofimovich with their colleagues has clear practical relevance to the effective teaching of second languages. For example, recognising the signs of stress can enable teachers to change the direction of conversations or reassure students, thus helping to eliminate the negative impact of anxiety on language learning. Similarly, the use of appropriate facial expressions by teachers could improve the learning experience. However, they believe that the most important consequence of their work may be a better understanding of non-verbal communication between individuals. In this context, they hope that the findings of their research will help not only students but couples and work colleagues to communicate more effectively.
The Way Forward
Professors McDonough and Trofimovich and their team are committed to extending this important work. They are particularly keen to expand their work from the laboratory to more natural settings. They are also conscious that some students may have an inherently more nervous disposition that makes differentiating between the stress caused by a challenging speaking task and the consequences of general anxiety, rather difficult. Addressing these issues will represent the next steps for the team in developing effective strategies to support those learning a second language.
MEET THE RESEARCHER
Professors Kim McDonough and Pavel Trofimovich
Department of Education
Professor Kim McDonough initially trained at the University of Michigan, where she graduated with a BA in Political Science and Studies in Religion in 1992. After teaching English in Thailand for four years, she continued her studies at Michigan State University, where she obtained a MA degree in Teaching English for Speakers of Other Languages in 1998. Finally, she completed her PhD degree in Applied Linguistics at Georgetown University in Washington DC in 2001. She started as an Assistant Professor at the Division of English as an International Language at the University of Illinois Urbana-Campaign the same year. In 2005, Professor McDonough moved to Northern Arizona University where she became an Associate Professor in Applied Linguistics. She joined the Department of Education at Concordia University in 2010, where she became a Professor of Applied Linguistics in 2015. While at Concordia University, she served as a Canada Research Chair in Applied Linguistics from 2011 to 2021. Her primary research goal is to identify how interaction facilitates second-language learning, and she is also interested in developing written and oral communication tasks that promote peer interaction in second-language classrooms. Her most recent research investigates the role of visual cues during face-to-face conversations.
Professor Pavel Trofimovich joined Concordia University’s Department of Education as an Assistant Professor in 2003 after earning a PhD in Educational Psychology (with a specialisation in second language acquisition) from the University of Illinois at Urbana-Champaign. In 2015, he was promoted to the rank of Professor. Outside the university, he has served as Associate Editor for Canadian Modern Language Review and Language Learning, and as Journal Editor for Language Learning in 2015–2019 and 2022–2023. His research focuses on cognitive aspects of second language processing and learning, the acquisition of second language pronunciation and speaking skills by children and adults, social aspects of second language communication, and the teaching of second language pronunciation.
E: Kim.McDonough@concordia.ca; Pavel.Trofimovich@concordia.ca
Professor Masatoshi Sato, Andres Bello University, Santiago, Chile
Dr Phung Dao, University of Cambridge, UK
Dr Dato Abashidze, Leibniz-Zentrum für Allgemeine Sprachwissenschaft, Germany
Dr Aki Tsunemoto, Kansai University, Japan
Canadian Social Science and Humanities Research Council
Canada Research Chairs Program, Fonds de recherche du Québec–Société et culture
K McDonough, R Lindberg, P Trofimovich, Examining rater perception of holds as a visual cue of listener nonunderstanding, Studies in Second Language Acquisition, 2022, 44, 1240–1259. DOI: https://doi.org/10.1017/S0272263122000018
A Tsunemoto, R Lindberg, P Trofimovich, K McDonough, Visual cues and rater perceptions of second language comprehensibility, accentedness, and fluency, Studies in Second Language Acquisition, 2022, 44, 659–684. DOI: https://doi.org/10.1017/S0272263121000425
R Lindberg, K McDonough, P Trofimovich, Investigating verbal and nonverbal indicators of physiological response during second language interaction, Applied Psycholinguistics, 2021, 42, 1403–1425. DOI: https://doi.org/10.1017/S014271642100028X
K McDonough, P Trofimovich, P Dao, D Abashidze, Eye gaze and L2 speakers’ responses to recasts: A systematic replication study of McDonough, Crowther, Kielstra, and Trofimovich (2015), Language Teaching, 2020, 53(1), 81–95. DOI: https://doi.org/10.1017/S0261444818000368
K McDonough, P Trofimovich, L Lu, D Abashidze, The occurrence and perception of listener visual cues during nonunderstanding episodes, Studies in Second Language Acquisition, 2019, 41, 1151–1165. DOI: https://doi.org/10.1017/S0272263119000238
K McDonough, P Trofimovich, P Dao, D Abashidze, Eye gaze and L2 speakers’ responses to recasts: A systematic replication study of McDonough, Crowther, Kielstra and Trofimovich (2015), 2018, Language Teaching, 53, 81–95. DOI: https://doi.org/10.1017/S0261444818000368
REPUBLISH OUR ARTICLES
We encourage all formats of sharing and republishing of our articles. Whether you want to host on your website, publication or blog, we welcome this. Find out more
Creative Commons Licence (CC BY 4.0)
This work is licensed under a Creative Commons Attribution 4.0 International License.
What does this mean?
Share: You can copy and redistribute the material in any medium or format
Adapt: You can change, and build upon the material for any purpose, even commercially.
Credit: You must give appropriate credit, provide a link to the license, and indicate if changes were made.
MORE ARTICLES YOU MAY LIKE
Professor Michael Ryan | What the Ugandan Response to HIV/AIDS Can Teach Us About Collaborative Governance
Persistent problems such as poverty, disease and hunger are of critical interest to organisation and management scholars. Developing countries often struggle with intractable social issues, including susceptibility to epidemics. The complexity of these challenges means it can be difficult for leaders to organise governance and ensure that resources and capabilities are effectively coordinated. Professor Michael Ryan looks at the case study of the HIV/AIDS response in Uganda, and asks how this can contribute to our understanding of public organisation and state capacity. In particular, he explores how Uganda was successful in using collaborative governance to manage the HIV/AIDS crisis.
Adipose tissue is more commonly known as body fat. Unlike white adipose tissue, which is linked to negative cardiovascular outcomes such as metabolic syndrome, brown adipose tissue is positively related to health and may reduce the risk of cardiometabolic diseases. Professor Aaron Brown and his team at the MaineHealth Institute for Research are working to understand brown adipose tissue regulation and explore its therapeutic potential. They have already made significant advancements through the application of optogenetics, a cutting-edge technique that harnesses the power of light to precisely manipulate specific cellular processes.
Who gets to be a theorist? What kinds of theoretical work get marginalised in academic research? And how does this oppression play out in the peer-review process? Dr Kimberly Kay Hoang is a Professor of Sociology at the University of Chicago. She has explored how difficult it is to get your sociology research published if you are not using research deemed to be legitimate by reviewers. She brings awareness to these issues and argues for change amongst scholars so that new forms of knowledge are not missed, especially regarding feminist, minority and racial theories.
The anonymous leak of the Panama Papers in 2016 revealed how the exceptionally wealthy (such as politicians, celebrities and business leaders) hide their money and exploit secretive offshore tax regimes. Dr Kimberly Kay Hoang is a Professor of Sociology at the University of Chicago, and after six years of research, hundreds of interviews and travelling 350,000 miles, she published Spiderweb Capitalism: How Global Elites Exploit Frontier Markets. She uncovered the mechanisms behind the movement of money into and out of Southeast Asia, and how that money travels all over the world.