Real-life Immersion to Learn a Foreign Language

Summary

In today’s global digital economy, transcultural competence and language proficiency is a must. But it is a hard skill to gain unless you have the ability to travel to other countries for long periods of time and practice the language in its original environment.

At Harvard, French Language 11 students now have the ability to experience Parisian life through the eyes of 4 locals in short, intensive first-person experiences. But not only. They can also interact with one of the character to practice their speaking and writing skills.

Made accessible directly from within Harvard FAS on-campus classrooms in Boston, this multi-part interactive virtual experience allows them to interact with the different characters in complete cultural immersion and gain more confidence in day-to-day exchanges in a foreign language.

Read also the great article published on the OpenAI Education substack here: https://edunewsletter.openai.com/p/immersive-language-learning-with

‍

‍

About Harvard Faculty of Arts & Sciences

​​The Faculty of Arts and Sciences is the historic heart of Harvard University, founded in 1636. It is the home of Harvard’s undergraduate program as well as all of Harvard’s PhD programs, serving more than 10,000 students every year.

The 40 academic departments and 30+ excellence centers of the FAS support a community unparalleled in its academic excellence across the broadest range of liberal arts and sciences disciplines, including language learning.

Objectives

It is often said that 95% of language learning programs fail at providing students proficiency and confidence in speaking a foreign language.

Yet, in today’s digital, globalizing economy, transcultural competences and language proficiency is a must.

The best approach consists in allowing students to travel to other countries for long periods of time and practice the language in its original environment. But can you achieve similar outcomes with virtual immersive fields trips?

How can virtual reality enhance foreign language students' ability to envision and understand the diverse and multifaceted nature of target cultures?

Nicole Mills, the director of language programs at Harvard started her journey with Wonda by creating her own virtual field trip in Paris.

"The goal of the virtual field trip was to give students the opportunity to “live” the stories of diverse Parisians through virtual immersion. They could see not only the architecture and cultural landmarks but participate peripherally in experiences that were unexplored and unexpected. The four Parisians provided virtual access to certain encounters that students may never come across during a visit or long‐term stay in Paris."

‍

Part #1: The Field Trip

French Language students now have the ability to experience Parisian life through the eyes of 4 locals in short, intensive first-person experiences taking place in one of Paris' most lively neighbourhoods.

By engaging in such a simulation of life in a foreign city, La République explores the construction, the evolution, and the complexity of Parisian identity through French and francophone texts, images, film, advertisements, articles, and songs -  through extra sensory elements that make them feel emotionally involved

Made accessible directly from within the classroom in Boston, this interactive VR experience allows them to interact with the different characters in complete cultural immersion and gain more confidence in day-to-day exchanges in a foreign language.

Production approach:

• the project team casted and hired four different Parisians from the same quarter (La République) to document and share the stories of their lives with a VR camera over the course of one month
• Parisians where trained for 2 hours to use the 360 Camera they were provided with.
• Extra Audio commentaries and transcripts were added afterwards

The Field Trip Structure:

• La République is a 40‐min VR experience consisting of four‐character narratives. Each narrative is a video diary of one of the four character's lives composed of six to ten single‐sequence 360° video segments of 1–4 minutes in length.

Delivery Approach:

• In multiple sessions, in classroom (or remote during Covid)
• in large group 20' discussions after individual 5 min immersions in the headsets
• in pair of students, where one explains what they see to the other one in French
  
  

“The opportunity to talk through what we were seeing in real time ... was helpful practice for speaking on the fly.” says a Harvard student.

Key Wonda features:

• Replay and fast forward custom buttons that allow users to navigate the VR experience at their own pace. Users may pause, replay, skip, or return to previous clips at any time.
• Interactive written transcripts that allow users to pause clips and access written transcripts by simply activating a “question mark” icon and cueing the content of the conversation or narration. Students were given the autonomy to use the transcripts as desired, but it was not required.
• Audio narration can be accessed through visual icons. By simply gazing at the visual icon, the user can trigger a brief narration and/or description of the point of interest. For example, while attending Aude's party in her studio apartment, users can access descriptions of the party guests (narrated by Aude; see Figure 4).
• Gamified experiences allow users to interact with objects in the virtual environment. In one of the diaries, for example, Benoît does a magic card trick and the user virtually plays with Benoit by choosing cards using the eye‐gaze interface.

Other Tools they used to produce the virtual field trip

• Ricoh Theta S 360 Digital Spherical Camera
• Tripod – large/mini

‍

Results

Immersion in virtual reality environments can unite language, place, and community as students experience highly contextualized real‐world environments in visual, auditory, and sensory modalities.

Based on this experience, the innovative immersive field trips can engage participants in unique and different ways:

• Expansion of classroom boundaries and connections to the “real world” with experience of new or novel places, practices, and events
• Meaningful opportunities for cultural immersion together with deep contextualization
• Multiple sensory modalities
• Deep connection with course content, alter perceptions and attitudes, & ability to learn complex information (Liu, Dede, Huang, & Richards, 2017)

For her students, the feedback was overly positive:

“Seeing the quarters, they looked very different from what I’d imagined by piecing together facts and photos.”

“I learned what they actually look and feel like. Even with pictures and reading of descriptions, I didn’t really feel the personality of the places until now.”

“Being able to see details that a movie would gloss over, I hear the sounds of everyday life and being able to choose what to look at as if we were there made for a more compelling and interactive experience.”

‍

Part #2: The Conversational AI Simulation

Extend the experience with Lee AI, the digital twin of the real 11-year old Parisian featured in the La République original simulation.

Students can enhance their experience by having a conversation with the AI-powered avatar of Lee, and discuss his habits, his family, friends, and passions through simple and authentic conversations.

Last but not least, students can also engage with Lee to help them write their composition. 

Here is the project deck explaining the approach:
‍