Teaching and Learning English in AR-Infused Worlds

Image credit: agsandrew/Shutterstock

If you spent the last decade living under a rock, you could be forgiven for not having heard of AR, VR, MR, or XR (augmented reality, virtual reality, mixed reality, and cross reality, respectively). More than commonplace abbreviations today, these four abbreviations alone represent a digital marketplace worth tens of billions of dollars in investments and market share profits worldwide.

Both artificial intelligence (AI) and machine learning (ML), long aided by deep learning neural networks that mine large and complex datasets to help with decision-making, have for several decades now tried to mix the real world (the quality or state of being real) and a virtual world (being such in essence or effect, but not in fact; near or implied) to augment the user’s interactive explorer-like experience of places, spaces, and events (Bower et al., 2014).

Defining AR, VR, MR, and XR

Augmented Reality (AR)

According to Merriam-Webster Online Dictionary (n.d.), the blending of virtual reality with objects in the real world is what gives AR its distinct character: AR supplements the real world with computer-generated graphics of objects or 3D models that appear to coexist in the same space as the real world. Some of the most notable AR terms and features include the following:

Auras: The medium created by AR platforms that brings to life an object, image, or location/landscape using sound, audio, video, animation, embodied virtual characters, augmented reality agents (or AuRAs), photos, or graphics. The use of advanced image recognition blending real-world with rich interactive content is called “Auras” (or AR experiences).

Geolocation: The geographical latitude-longitude location of a user or an internet-connected computing device using an IP-to-location database.

Marker: A black and white square printed object for AR to scan; AR-markers are predefined visual cues that trigger the display of the virtual information.

Marker-Based AR: Employs static images mobile devices scan to trigger and display the computer-generated content; marker-based apps are dependent on image recognition technology.

Markerless AR: Provides location detection and helps the discovery of places; GPS, digital compass, and accelerometer combine effortlessly to add location information on the screen about the objects the device’s camera sees; markerless apps use GPS to locate places nearby or to offer directions.

NFC: Derived from RFID (Radio Frequency IDentification), Near Field Communication, a wireless data transfer communication technology, is a standards-based low speed, short-range connection between two compatible electronic devices (phones, tablets, laptops, etc.) allowing secure transactions and digital content exchange with a touch (e.g., between a phone and a payments terminal). Google Wallet and Apple Pay both use NFC tags to trigger the programmed action.

Overlay: The content of images, videos, 3D animations, and sounds that is superimposed upon the trigger image allowing users to scan the trigger image in the real world to view the “Auras,” add actions to each, and interact with the overlay digital content and information via a mobile device.

QR Code: Quick Response Code, a machine-readable optical label (or 2D digital matrix barcode image) holding information/data about a specific item, allowing the user to encode information instantly using their phone camera or reader app. QR Codes can include text, files, and links to videos, websites, social media, and the like. Today, QR codes are native to most smartphones and no longer require users to first download a separate app to scan QR codes. (See YouTube for a tutorial on Creating and Using QR Codes in the Classroom; see also QR Codes 101.)

SnapTags: Exhibiting the functionality of the QR Codes, SnapTags are scannable barcodes that can turn any image (e.g., logo, special icon, marketing campaign) into a barcode that surrounds the image with a “code ring,” a black-and-white ring with dots that blends seamlessly into any design. Scanning the SnapTag or taking a picture and sending it to a designated number as a text message enables access to the information on the ring-shaped code with the brand logo in the center.

Tag: A type of bar code similar to QR Codes that, when scanned by a Tag reader, links the user to online content.

Trigger Image: The image, text, or object (a jpg or png) users scan to engage with its information.

How These Features Are Used

Think of the built-in and easy-to-use Snapchat design filters (and lenses and stickers), which can be used in a casual way to augment your photos and videos. These virtual objects often contain auxiliary information tags, such as video, audio, sound, illustrations, images, animation, web links, and text, which are, in turn, digitally superimposed upon or composited with the surrounding real-world environment. Users interact with both “real” and “digital” worlds in real time and can distinguish clearly between them by using a smartphone, tablet, augmented reality glasses (see Google Glass 2), or other mobile or wearable device (see Microsoft HoloLens).

For example, the worldwide craze of the Pokémon Go app (an AR mobile game for iOS and Android devices) in the summer of 2016 is a prime example of such a simultaneous virtual-real world experience. To receive digital content overlaid onto the real world through visual recognition, users can launch an AR app and focus the device’s camera on a QR code with a marker for AR to scan, tap the smartphone against the NFC tag for the desired action to take place, or snap a photo of the SnapTag.

AR relies heavily on geolocation to display its digital content: The combined real and virtual image appears fixed in space, and the computer-generated sensory input, such as graphics, video, sound, or GPS data are entirely mobile, fluid, and scalable. The only constant in AR is the interconnection of the two coexisting environments—the real environment the user views and the virtual environment the computer generates. In distinct ways, AR supplements (it does not replace) reality and is regularly used in interior furnishings, games, navigation apps, ecommerce, tourism, architecture, construction, training and education, and more.

Notable AR Platforms

Following are some of the most notable augmented reality platforms:

• 3DAR: studio specializing in content creation, 3D animation, live action, and postproduction for the advertisement and film industry

• ARCore: a software development kit developed by Google that allows for AR applications to be built

• ARIS: an open-source platform for creating and playing AR experiences on iOS devices

• ARKit: integrates iOS device camera and motion features to produce AR experiences in your app or game

• AR ToolKit: an open-source computer tracking library for creation of AR applications

• Blippar: a simple drag and drop AR creation tool for App and WebAR

• Beaconstac (formerly HP Reveal): a comprehensive QR Code solution delivering custom AR experiences.

• Kudan: cross-platform development of marker and markerless AR

• Lens Studio: platform for creating, publishing, and sharing AR experiences for Windows and Mac

• Mixare: a free open source AR browser for Android and iPhone

• PlugXR: cloud-based AR platform with features to create and publish advanced AR apps and experiences

• ROAR AR: a self-service platform for building AR experiences in a few clicks

• Spark AR Studio: a platform for creating and sharing interactive AR experiences with or without code

• String AR: a free AR app (see how it can be used at geteducreative.com)

• Vuforia Engine: a software development kit for creating AR apps

• WebAR: a web development platform for creating app-based AR

• WebXR: delivers immersive experiences in VR and AR that are compatible with all modern web browsers on PC and mobile

• Wikitude: provides tools for businesses to create impactful AR experiences and solutions across industries

Virtual Reality (VR)

In contrast to AR, VR (virtual reality, virtuality, or immersive multimedia) is a computer-generated, multisensory information program that simulates physical presence in imagined, nonphysical 3D digital “spaces” and “places” able to track users in real time. VR thus refers to a high-end user interface (human-to-computer) that involves real-time simulation and interactions in an artificial 3D multimedia format through multiple sensorial channels (sight, hearing, touch, and other tactile-kinesthetic sense perceptions). Users interact by wearing stereoscopic goggles or a head-mounted display helmet (e.g., Oculus Rift, HTC Vive), database gloves (sensors that track hand movements), and other hand-held miniaturization devices for input.

Mixed/Merged Reality (MR) and Cross Reality (XR)

The merging (blending) of real worlds and virtual worlds that includes both real and computer-generated objects is MR or MxR (merged or mixed reality, respectively). It combines aspects of AR (a semidigital experience in the real, physical environment) and VR (a fully digital experience in a computer-generated, 3D environment) to produce new environments and visualizations, where physical (real) and digital (virtual) objects coexist and interact in real time. A headset enables users to interact with both in real time (see Microsoft HoloLens).

Similarly, XR (cross reality) is content using emerging technologies, such as AR, VR, or MR. It is also hardware such as Google Tango, a high-end smartphone AR platform Google shut down in late 2017 and replaced with ARCore in March 2018.

Benefits and Considerations

A large body of research has already shown that AR can indeed be applied to teach a variety of academic subjects and English to learners, from the youngest pupils to adult learners (see Further Reading). Because the use of smartphones and tablets in the teaching and learning process is both portable and ubiquitous, AR can be used resourcefully to afford benefits not easily duplicated. Among these, AR...

• provides access to distance education

• activates strategic planning across subject areas and within language levels

• engages different learning styles

• optimizes interactive learning

• enhances social integration of learners

• creates independent and collaborative learning environments

• boosts engagement and motivation

• sparks curiosity and imagination

• stimulates creative thinking

• advances authentic contextual practice

• builds understanding between digital and real worlds

• delivers motivational-emotional rewards

• offers real-time feedback

Irrespective of learning environments, it is the combination of physical affordances (e.g., look and feel of real objects; size, shape, texture, color, weight; environment location, angle, positioning) and virtual affordances (e.g., interface design, look of virtual objects, copy of real objects; three-dimensional space; scene-setting; placement of digital objects within the real world; video-animation “auras”) that redefines and repurposes our lived experience with such advanced technologies.

Practical Uses of AR

Within AR learning environments stimulating increased interest and confidence in problem-solving, collaboration, and knowledge creation, students can, for example, effortlessly harness the technology that affords them a great many opportunities to explore settings in which authentic content and language learning is made both relevant and consequential.

Moreover, learning can be transmitted via any combination of text, graphics, graphic art, images, illustrations, sound, animation, audio, and video on analogue print objects, such as books, magazines, or flash cards, to effectively communicate dynamic concepts and ideas to users delivered by a smartphone or tablet or other mobile device. (See YouTube for an HP Reveal app tutorial.) Some of these practical scenarios for AR research-and-praxis include developing skill and knowledge to do the following:

• Bring Objects, Places, and Models Into the Classroom: Virtually transport objects into the classroom that would be difficult to accomplish in real life: animals that teach numbers or the alphabet; time lapses that explore historical places, spaces, and events; and 3D models that peer into the details of building architecture, weather systems and patterns, ecology and ecosystems, and plant life. Arloon Plants AR, for example, focuses on plant adaptations in five ecosystems (Taiga, Desert, Mediterranean, Forest, Steppe) and includes animations about plant ecosystems, processes, and classification. Foundational science concepts spring to life readily with interactive animation.
• Explore Abstract or Distant Concepts, Places, and People: Bring AR content to life and interact with it in varied ways: Visualize and experiment with abstract concepts, observe the frog life cycle in Froggipedia, or explore wildlife, people, and the landscape in WWF Free Rivers.
• Explore Chemistry and Human Anatomy: Cover topics and lesson plans in chemistry and human anatomy via Elements 4D and Anatomy 4D apps, respectively (see also Insight Heart, which explores human heart anatomy and cardiovascular conditions with detailed 3D medical animations).
• Make Geometry Tangible: Build and measure 2D and 3D scalable geometric shapes and solids via Shapes 3D Create Geometry AR, construct prisms and pyramids, and change units between metric units. Create 3D shapes and equations in GeoGebra AR and view 3D models in every angle, distance, or scale desired to improve understanding of abstract, spatial geometric concepts, and place 3D math objects on any surface.
• Journey Through Time and Space: Experience an interactive journey through the birth and evolution of the universe via Big Bang AR: Discover space and time, witness the formation of stars and our solar system, and even hold the Earth in the palm of your hand.
• Interact With Museum Exhibits: Take virtual trips to museums around the world to explore ancient artifacts while discovering societies, cultures, and treasures via Civilisations AR by the BBC: Uncover the secrets of ancient Egypt; explore the layers beneath Renaissance masterpieces; see inside an Egyptian sarcophagus; translate the iconic Rosetta Stone hieroglyphics; and move, scale, and rotate the collection of more than 30 historic artifacts and cultural treasures.
• Tap Into Creativity and the Arts: Color pages via Quiver (a 3D-coloring app); build inventive settings with drawings, photos, and video to share interactive short stories via AR Makr; and create personalized AR-content and lesson plans. Both Aurasma and Blippar provide easy-to-use mobile and web application tools to facilitate interactive content creation.
• Build Your Own Augmented Reality: Make productive use of Blippar, an AR authoring option used widely in publishing, to employ pictures and markers as triggers.
• Engage Classroom Displays With Multisensory Learning Experiences: Create word walls, interactive posters, bulletin boards, and self-tests of vocabulary, phrases, and concepts to measure learning; self-directed or task-based AR educational games; game-based learning and gamification; scavenger hunts and adventure quests.
• Create Interactive Educational Games: Participate in scavenger hunt games with an AR twist via Waypoint EDU, learn about ancient wonders (Stonehenge, Pyramid of Giza), witness inventions (telephone, light bulb), and create your own hunt games and quizzes.
• Create Location-Based Games and Stories: Employ free user-friendly, open-source AR game editors like ARIS (Augmented Reality and Interactive Storytelling) or TaleBlazer to create and play Pokémon Go–style location-based mobile games, tours, and interactive stories for players to experience and promote language learning on iOS or Android devices.

Not only will such AR scenarios lessen students' language learning anxiety, but even more importantly, perhaps, the acquisition of information and skills will be augmented and integrated in powerful new research-and-praxis ways. In the end, (language) learning is triggered, knowledge is attained, and memories of having "lived" both are not soon forgotten, especially when students are offered relevant opportunities to tell their own AR stories in their own individual ways. (For an extensive list of digital tools and platforms, see Liontas, in press; also visit Digital Storytelling.)

Moving Forward in the New Landscape of AR Education

The combination of technology device, overlay, and trigger (or marker via the AR app that activates the overlay, the action that plays within the camera view, such as image, video, GIF, or 3D model) makes AR simple to use and interact with objects in virtual spaces. Interactivity and engagement are by far AR’s two greatest attributes. Simply put, AR is an intervening “reality” that opens new avenues for learning while interacting with the real world.

Yes, AR has some drawbacks:

• It does require an app download for iOS or Android devices.

• It may work primarily with static media.

• File sizes can be large and slow down the viewing experience pending internet connection and size limit for content that can be uploaded.

• Voice input can be compromised (especially in noisy environments).

• Display size may be limited (and displays need to be held for extended periods of time).

• AR content development may well be too time-consuming, at least initially.

• There may be copyright issues related to images/videos used.

• There may be size limits for content that can be uploaded.

• Archiving and providing access to dynamic content may cause some unwarranted headaches.

However, AR developers and users willing to confront these challenges can nonetheless orchestrate AR-infused worlds wherein learning independently is maximized, content access is liberated, and engagement through purposeful utility or edutainment is optimized. Armed with easy-to-follow online or video tutorials, all such perceived challenges are easily surmountable with some investments in time and effort. And just as AR is progressively used in retail to usher product discovery and awareness prior to virtual product trial and personalization, content-area teachers and English language professionals alike can indeed employ AR to provide next learning virtualization for increased discovery, trial, engagement, personalization, and conversion of (language) content data into powerful learning AR formats, defying conventionality and instructional monotony: one AR-experience at a time, one reality superimposed—digitally, holographically, inventively.

References

• Bower, M., Howe, C., McCredie, N., Robinson, A., & Grover, D. (2014). Augmented reality in education: Cases, places, and potentials. Educational Media International, 51(1), 1–15.
• Liontas, J. I. (in press). Attaining knowledge of idiomatics in the age of coronavirus and beyond. In J. Perren, K. Kelch, J-s. Byun, S. Cervantes, & S. Safavi (Eds.), Applications of CALL theory in ESL and EFL environments. IGI Global.
• Merriam-Webster (n.d.). Augmented reality. In Merriam-Webster.com dictionary. Retrieved December 14, 2020, from https://www.merriam-webster.com/dictionary/augmented%20reality

For Further Reading

• Blyth, C. (2018). Immersive technologies and language learning. Foreign Language Annals, 51(1), 225–232.
• Bonner, E., & Reinders, H. (2018). Augmented and virtual reality in the language classroom: Practical ideas. Teaching English with Technology, 18(3), 33–53.
• Godwin-Jones, R. (2016). Augmented reality and language learning: From annotated vocabulary to place-based mobile games. Language Learning & Technology 20(3), 9–19.
• Ibáñez, M.-B., & Delgado-Kloos, C. (2018). Augmented reality for STEM learning: A systematic review. Computers & Education, 123, 109–123.
• Perry, B. (2018). ARIS: A tool to promote language learning through AR gaming. CALICO Journal, 35(3), 333–342.
• Richardson, D. (2016). Exploring the potential of a location based augmented reality game for language learning. International Journal of Game-Based Learning, 6(3), 34–49.
• Scrivner, O., Madewell, J., Buckley, C., & Perez, N. (2019). Best practices in the use of augmented and virtual reality technologies for SLA: Design, implementation, and feedback. In M. L. Carrió-Pastor (Ed.), Teaching language and teaching literature in virtual environments (pp. 55–72). Springer Nature, Singapore Pte Ltd.

---

Dr. John I. Liontas is the director of the Technology in Education and Second Language Acquisition (TESLA) doctoral program at the University of South Florida, where he teaches masters and doctoral courses in ESOL, SLA, idiomatics, and emerging digital technologies. He is an active member in (inter)national learned societies, a distinguished thought leader, a multiple award-winning author, a researcher, and a practitioner. He is also the editor-in-chief of the award-winning encyclopedia, The TESOL Encyclopedia of English Language Teaching (Wiley, 2018), the first print and online encyclopedia for TESOL International Association since its founding in 1966.