StoryToGo

Exploring storytelling in its many forms in this digital age

#StoryToGo

Conceptualising an Extended Reality Open Education Guide

by 6 Comments

As we step into 2024 and I wrap up my ETEC 580 Directed Studies and Master of Educational Technology, I want to take a moment to conceptualise the Extended Reality Open Education Guide that I have begun to build; to give it shape, intention and create a road map for it, as well to seek feedback from you, our readers, as I continue to further my journey, experiments, and goals in extended reality (XR) beyond my Masters. As I previously wrote in Exploring XR Development with my Final Master of Educational Technology Directed Studies, the work I have done through the MET Program at UBC is a beginning to this work in XR development.

Here’s where I am at present in mapping out where I am going with this work:

XR Open Education Guide for StoryToGo

As I continue to develop an XR Open Education Guide on StoryToGo, much as I did with the Web Monetization Guide, there are 4 main areas that I will be focusing on, developing, and building upon over time:

  • XR Case Studies
  • XR Development Experiments and Process Blogging
  • XR Development Series of Mini-Courses
  • Addressing Accessibility in Extended Reality

XR Case Studies

The goal of the Extended Reality Case Studies is to explore different extended reality experiences and applications to discover what resonates with myself and my team and to identify the challenges. By writing these up as case studies, the hope is to create a resource for our team in developing extended reality experiences, and to share that resource more broadly with other independent creatives, educators and students wanting to explore extended reality. Secondarily, these XR Case Studies will also be used as resources and examples within the XR Development Series of Mini-Courses.

At present, these XR Case Studies are focused around augmented reality (AR) natural and cultural history experiences and virtual reality (VR) health and wellness experiences, as these are the areas that my team and I are currently developing extended reality projects. Over times, we will broaden these case studies to other areas of interest.

Extended Reality Case Studies

As my team’s and my current priority is on AR natural and cultural history experiences, the current case studies include:

I am also writing up a case study on the TaleBlazer AR Game at the Royal Botanical Garden’s Rose Garden at present.

Other XR Case Studies on the current ‘to write’ list, include:

  • Notre-Dame de Paris, The Augmented Exhibition
  • Plank Walk VR
  • Pain Distraction VR
  • Concussion Therapy VR

As the XR Case Studies grow in number on StoryToGo, we will create an Experiential Reality Library, like the Web Monetized Content Library that we’ve begun to create.

XR Development Experiments and Process Blogging

As an indie creative who likes experimenting with emerging ways of crafting stories, I find it helpful to experiment with the technology for building those stories. That way even if we contract a larger team on our builds, I know what I am talking about and know what is possible. Often as I dive into such experiments, I also learn that building stories in these newer ways with emerging technologies is not so daunting, cost prohibitive, and inaccessible as people think. Thus, as I create and experiment with extended reality development, using different extended reality development tools and platforms, I intend to create process blogs walking people through the ‘how to’ of my experiments creating with different technologies.

The purpose?

  • to create a running log / record of the process to pull from for the XR Development Series of Mini Courses;
  • to engage others around this process, and crowdsource ideas and solutions; and
  • inspire others to experiment with their own XR builds, and provide them with a resource to helping them do so.
Extended Reality Development Experiments and Process Blogging

Currently my goals for my own projects involve Unreal Engine and motion capture technology. This, however, is not the most accessible entry point for educators and students wanting to begin their own experiments with XR, both in terms of the learning curve and in terms of the expense of the technology needed for. As such, I decided for the purposes of creating a more accessible entry to extended reality for teachers and students, that I want to begin with some more accessible extended reality builds / development ideas, and then build towards the more involved pieces using UnReal Engine and Unity, as part of my long term, on-going work.

What this means, is that my initial AR build will be with TaleBlazer to create some early development options within the open Extended Reality Course that are designed for teachers and students, and then later build Units on developing with UnReal Engine and Unity, as well as other more accessible extended reality development solutions, like Stornaway.io, CoSpaces, Halo AR, polycam, MyWebAR, and 8th Wall. If you have other suggestions of XR development tools and platforms that my team and I should be experimenting with, please let me know about them in the comments on this post.

In terms of the extended reality projects that my team and I have been developing, they include:

  • WWII stories from my friend Manami Calvo (Saito)’s family from their experience as Canadians of Japanese ancestry, living on the West Coast in WWII
    • our plan for this is to create some initial experimental ‘art installations’ using Web AR, that are accessible via smartphones and tablets
    • we plan on creating these utilizing animated ‘ghostly figures’ created with Unreal Engine and a motion capture suit, along with historic photos and video, as well as recorded interviews
    • we will then use those art installations to go after proper funding to further development

  • health / wellness VR Experiences
    • our plan is to start with a kids yoga class in VR that can be used in hospitals, as well as at home
    • create in Unreal Engine with a motion capture suit
    • the yoga will be lead by Lori Yearwood (wearing the motion capture suit) and will involve her transforming to different plants and animals with different poses
    • the children will also be able to select the background world (from a variety of calming choices) for their yoga experience

XR Development Series of Mini-Courses

To make things as easy as possible for other indie creatives, educators and students to begin to explore and develop their own extended reality projects, I intend to take what I have learned through my extended reality research, case studies, experiments and process blogging, and build a series of Extended Reality Mini-Courses for people to use as a guide in creating their own extended reality projects. The goal here being to remove the fear of extended reality being a far reaching, inaccessible goal by building their understanding of extended reality, getting them experimenting with it and envisioning the possibilities, and sharing with them a variety of pathways to creating extended reality, including step-by-step guides.

Extended Reality Development Courses

In developing a series of Extended Reality Mini-Courses, I am exploring other Extended Reality Learning Resources to see what is currently available to those wishing to explore extended reality further, so as best to identify a different, unique approach to the subject and include any beneficial resources that are currently missing.

The Extended Reality Learning Resources that I am currently delving into, include:

Pre-Existing Extended Reality Courses

In approaching the extended reality course development for the StoryToGo Classroom site, I have come to the conclusion that XR development is more manageably approached as a series of mini-courses. Currently, I am thinking in terms of building the following three courses:

  • First Steps in Extended Reality Development – An Introduction
  • Extended Reality Development
  • Extended Reality User Testing and Refining

The talk I gave for the class at the American University in Cairo follows the layout for the first mini-course: 

My current breakdown for ‘First Steps in Extended Reality Development – An Introduction’ will include:

  • What Extended Reality Is
    • types of Extended Reality
    • applications of Extended Reality
    • delivery of Extended Reality and Devices

  • Extended Reality Case Studies
    • an informal introduction to user testing
    • creating mini case studies in user testing extended reality experiences, and recording stars and wishes from the experiences
  • Developing an Extended Reality Idea of Your Own
    • Why XR?
      • what type of XR?
      • is this the strongest choice?
      • how do you wish to deliver? to which devices? why?
    • Do you need to create all at once or can you create it in accessible steps?
    • Steps in creating
    • Creating a roadmap

Then the Extended Reality Development Course will be a continual work in progress, adding different Units, covering how to build XR with different extended reality development solutions, starting with TaleBlazer.

Extended Reality Mini Course to Develop for StoryToGo

If you have suggestions for the structure of or things I should be including in the XR Development Series of Mini-Courses, please share your thoughts in the comments.

Addressing Accessibility in Extended Reality

While not a standalone piece on my brainstorming document, for those of you who have begun to read our XR Case Studies, you may have recognized that accessibility in extended reality is central to my team’s and my explorations and development goals. This is very much driven by various disabilities and health and wellness issues that my team and I have experienced, but is also an aspect of extended reality design and user testing that has not been discussed and addressed as well as it should yet.

For now, my team and I intended to address Accessibility in Extended Reality throughout our case studies, process blogging, and course materials. I also had an invitation to come on as an advisor on accessibility to UBC’s Emerging Media Lab, after I brought accessibility and my perspective on with a few of the students at their Autumn Showcase. This is an invitation I intend to follow up on this year, with the hope that between that and my own team’s process thinking through accessibility in extended reality, as we share extended reality case studies, work on our own extended reality development and process blog on it, and create our Extended Reality Development Series of Mini-Courses, that we also begin to build a standalone piece or pieces that address accessibility in extended reality. Not sure what that will be yet, but if you have suggestions for this, I’d love to hear about them in the comments.

Your Thoughts?

I’d love to hear your thoughts in comments below on the direction I am taking with this Extended Reality Open Education Guide. If you have suggestions for future XR Case Studies, on XR Development Tools and Platforms to try, the structure of or things I should be including in the XR Development Series of Mini-Courses, or ways to further address Accessibility in Extended Reality, please share them in the comments. Thank you!

Also, if anyone wishes to become involved in any of the above endeavours with myself and my team, please reach out and we can chat further.

Thank Yous

Thank You!

The work that I have begun here has been inspired by and is evolving, thanks to:

  • Dr David Vogt for encouraging my dreaming, entrepreneurial thinking, and edtech product design and development;
  • Dr David Porter in exploring open education and searching for sustainable funding pathways for;
  • Saeed Dyanatkar for seeing my potential in dreaming up different pathways for storytelling and education, always making time for me when I had questions, and making me feel welcome and valued whenever I wandered into the Emerging Media Lab;
  • Dr Kyle Stooshnov and Juliana Loh for creating a framework for my first steps into thinking about XR development and experimenting with 360 video;
  • Dr Jennifer Jenson and Dr Suzanne de Castell for fostering my game design thinking;
  • Dr Heidi Janz, Dr Michelle Stack and Dr Paul Hamilton for helping me to process my newer disabilities, frame disabilities and myself as disabled in my storytelling, and begin to understand how to advocate for disability and accessibility in education (and feel empowered to do so, understanding it as a strength, as oppose to something to hide);
  • Yvonne Dawydiak for demonstrating the alternative pathways to navigating UBC, approaching teaching creatively in a hands-on way, and an empathy driven approach and practice;
  • Lori Yearwood for her vision in beginning to create case studies here on StoryToGo to act as resources for our students, workshop participants, clients and partners, and the broader independent creative and education community – for encouraging me to begin to share my disabilities in our storytelling and work, and sharing her own wellness challenges as we began to build our health, wellness and accessibility workshops, resources, and storytelling – and for always being a willing and enthusiastic supporter and co-creator on the ideas that wander through my mind; and
  • Manami Calvo for seeing the potential for sharing her family’s WWII stories in AR, joining me in crafting experiments around, and in trusting me with her family’s stories.

Also an enormous thank you to the Interledger Foundation for helping fund my research and work on Sustainable Funding Solutions for Creatives and Educators, and to CNIE – RCIÉ and BCIT who have helped fund my research and coursework throughout my Masters.

Thank you all, for all your help, support, direction and inspiration!

Seeing the Invisible Augmented Reality Art Exhibit – an Extended Reality Case Study

by 4 Comments

In this second case study in our series exploring extended reality (XR), we examine the Seeing the Invisible Augmented Reality (AR) Art Exhibit, as exhibited by the Royal Botanical Gardens. This AR art exhibition featuring the art of 13 artists from around the world was initiated by the Jerusalem Botanical Gardens and displayed concurrently at 12 Botanical Gardens worldwide.

Project:

Seeing the Invisible Augmented Reality Art Exhibit

Initiated By:

Jerusalem Botanical Gardens in partnership with Outset Contemporary Art Fund, with the support of the Jerusalem Foundation

Curated By:

Hadas Maor and Tal Michael Haring

Host At:

Royal Botanical Gardens (RBG)

Type of Experience:

Augmented Reality (AR) Outdoor Art Exhibition

Device for Experiencing:

Smartphone or Tablet, preferably with headphones

Link to Project:

https://seeingtheinvisible.art/

Date Partook in the Experience:

September 18, 2022

Country of Origin:

Israel

Country Where I Experienced:

Canada

Reflections On My Experience

Interview Between Tucson Botanical Gardens Executive Director Michelle Conklin & Seeing The Invisible Curators Hadas Maor & Tal Michael Haring

The Project:

In 2021, as the world continued to face the pandemic and many art galleries were closed to the public, an idea was fostered out of the Jerusalem Botanical Gardens to host an outdoor art exhibit, but not in the usual sort of way. Rather an invisible art exhibit, at least to the naked eye, and one that was concurrently displayed in 12 gardens worldwide with the same pieces of original art. “How?”, you ask. With augment reality (AR), creating the first such multi-location, simultaneous art exhibition of its kind with a number of artists, many of whom were new to AR, creating their first pieces in augmented reality.

Some of the goals of this exhibition were to:

  • give visitors something new to enjoy in gardens at times of year when plants may not be blooming
  • complement the natural setting with the AR art, encouraging visitors to experience it in a new way
  • give people a way to enjoy art in a shared, yet outdoor setting
  • encourage visitors to the gardens to engage with the gardens and with the art
  • collaborate with other gardens worldwide
  • create an exhibition without disturbing the gardens themselves, and keeping the carbon footprint to a minimum

The art exhibition was also designed to address shared themes of “nature, environment, and sustainability, exploring the boundaries and connections between art, technology, and nature. Both bleak and hopeful, each artwork offering a unique perspective on unresolved issues, creating thought-provoking, experiential, and contemplative spaces for viewers to immerse in.”

The artwork and artists crafting this narrative included:

Despite the art exhibition taking place concurrently in a number of different gardens, the experience is different in each garden, as the works are augmenting the unique surroundings and context of each garden.

Why Augmented Reality?

Building and purchasing sculptures and other art installations is a pretty major undertaking for a botanical gardens, and a travelling exhibition can generally only visit one garden at a time. While not without a cost, by making this art exhibition simultaneously available in a number of botanical gardens, this makes the cost of such an art exhibition less prohibitive. The fact that this art exhibition is accessible through augmented reality via digital devices, makes it possible to ‘install the pieces’ temporarily in multiple gardens at the same time.

The initial launch of this AR art exhibition during the pandemic was well timed to offer an alternative outdoor way to enjoy art, outside the enclosed space of an art gallery, and further encouraged visitors to think about botanical gardens as arts spaces, as well as nature spaces.

The curators of Seeing the Invisible talk about the importance of this AR art exhibition being phygital, a blending of a digital experience with a physical one. Thought was put into where each artwork was placed in each garden to enhance both the artwork and the garden, as well as take visitor safety and the protection of the planted exhibitions into consideration. The AR artwork, through the Seeing the Invisible app, was geotagged to specific locations in the gardens they were exhibited in, and visitors could only experience the artwork in that location, and only in those gardens. Thanks to the phygital, the experience was unique in each garden.

The Augmented Reality Experience

The Seeing the Invisible artwork is experienced at the gardens hosting this AR art exhibition through visitors’ smartphones and tablets via a GPS triggered app.

Viewing AR art, Dawn Chorus, through a smartphone.

To see and experience these dynamic and engaging pieces of artwork, people visiting the gardens hosting this AR art exhibition need to:

  • Before visiting a Gardens hosting the AR art exhibition:
    • Download the Seeing the Invisible App to the smartphone or tablet with cellular capabilities that they will be using at the AR art exhibition.
    • Allow the app access to device’s camera and microphone.
    • Fully charge the device before visiting the gardens.
    • Take earbuds or headphones compatible with the device to the gardens with you.
  • At a Gardens hosting the AR art exhibition:
    • If you don’t have a smartphone or tablet, you can borrow one on site.
    • Follow the map in the app to the different pieces of AR art within the gardens.
    • Follow the Seeing the Invisible App’s instructions at each piece of AR art, scanning the ground where the piece of art has been virtually installed until the art virtually appears.

The Royal Botanical Gardens shares further tips on enjoying Seeing the Invisible Art Exhibition in their gardens, here.

Once the art virtually appears before you, visitors are encouraged to engage with it and fully immerse themselves in it, by listening through their headphones to the experience, and walking around the artwork and even into it. Visitors can also take pictures with the virtual art and read more about the art through the app.

Each piece of art, lends to different forms of engagement. For instance with:

  • Gilded Cage: You can walk into the labyrinth of cells.
  • Dawn Chorus: Birds swoop around you as you are drawn in by the music from the piano.
  • Biome Gateway: You can walk into the temple cave, and discover a portal to walk through and into a parallel landscape.
  • Forget Me Not: By moving toward the flower arrangement, you trigger a gun to propel a bullet through the flower arrangement causing it to explode outwards. Once the explosion has occurred, if you walk around the vase, you will hear three different scholars discussing the flower arrangement.
  • Pneuma: By walking into the sphere, you see the gardens around you through the distortion of being inside a bubble, while the sound of breathing acts to change your own breathing pattern while experiencing.

Morphecore Prototype AR even inspired a dancer to choreograph their own piece to dance along with. Other host Gardens similarly witnessed many of their visitors dancing along with the Morphecore Prototype.

My Reflections on this Augmented Reality Experience

One of my goals in exploring different extended reality experiences is to discover what excites me in these experiences and what I find challenging, in order to reflect on what could create even richer and more accessible extended reality experiences.

Here are my thoughts on the Seeing the Invisible AR Art Exhibition, from when I visited it in September of 2022 at the Royal Botanical Gardens.

Signs welcoming visitors to the Seeing the Invisible AR Art Exhibition at the Royal Botanical Gardens.

The Extended Reality Magic

This is a clever idea to get visitors exploring the gardens through a new lens, allowing the augmented reality art to spark their imagination in a new way, to perceive the gardens differently, and to provoke thought pertaining to nature, technology and art.

Some art, like Dawn Chorus, brought pure joy of the fantastically, magical whimsy that augmented reality brings with it, further adding beauty to the surrounding gardens.

AR birds flying around an augmented reality piano in a real world garden.
AR birds flying around an augmented reality piano, care of Dawn Chorus, in a real world garden.

Other art, like Pneuma, made you feel as though you’d stepped out of reality and were peering back in at it.

AR Art looking back at the gardens through the distortion of the augmented reality bubble created by Mel O’Callaghan's Pneuma.
Looking back at the gardens through the distortion of the augmented reality bubble created by Mel O’Callaghan’s Pneuma.

While other art, like Gilded Cage, got you questioning our problematically omnipotent and controlling relationship with nature from the perspective on an artist who was imprisoned.

AR art, Ai Weiwei's golden Gilded Cage, casting augmented reality shadows, by a park bench in a real world garden.
AR art of Ai Weiwei’s golden Gilded Cage, casting augmented reality shadows by a park bench in a real world garden.

Best of all, the gardens discovered that people of all ages wanted to engage with the art, even dancing along to Daito Manabe’s Morphecore Prototype AR.

A real world dancer at Jerusalem Botanical Gardens dancing alongside Daito Manabe's Morphecore Prototype AR.
A real world dancer at Jerusalem Botanical Gardens dancing alongside Daito Manabe’s Morphecore Prototype AR.

Logistically the GPS triggered app is a good idea to create a curated experience within each garden, and to host this simultaneously between different gardens.

Adding in the ability for visitors to be able to photograph the AR art through the app, added in the ability for visitors to have fun engaging with the art. This also added to crowdsourced storytelling around the exhibit, which I can only imagine was beneficial to marketing the exhibition.

Visitors at a number of gardens playing with Pneuma selfies.
Visitors at a number of gardens playing with Pneuma selfies.

I’d love to experience future such AR art exhibitions at the Royal Botanical Garden, as well as to experience the Seeing the Invisible Art Exhibition in different gardens to see how the experience differs between the gardens with the same art.

Current Challenges (as of 2022 Experience)

Excited to experience the Seeing the Invisible Art Exhibition at the Royal Botanical Gardens in Burlington, Ontario, my niece and I followed the instructions and downloaded the Seeing the Invisible App to our smartphones before heading to Hendrie Park to enjoy the exhibit together.

Seeing the Invisible at the Royal Botanical Garden's Hendrie Park.

The only problem, when we got there, the Seeing the Invisible App didn’t work on either of our smartphones, despite both having compatible technology. Not sure if this is the reason why, but one woman told us the issue was due to the strength of the wifi signal from your data provider.

No problem, we queued to borrow one of tablets that the gardens had available for visitors that needed it.

Borrowing a loaner tablet at the Royal Botanical Gardens.

While the borrowed tablet worked, this now meant that we were now sharing a device, our headphones didn’t work in the borrowed device, and the borrowed tablet had a screen protector over it, that made the AR more difficult to see, especially in sunny areas. Added to this, almost every piece of AR art was placed in a sunny area, and it was hot, so not only were we battling with the light to see the AR art through the screen, but needed shade for respite for the sun.

I assumed that this user experience oversight was down to this being the Royal Botanical Garden’s first AR Exhibition and them not being familiar enough with the technology to understand how to create a positive user experience, but it turns out it was actually the curators that mapped out where each piece would be placed. Maybe the oversight of the sun was made, as due to the pandemic, the curators did not visit Hendrie Park before deciding where to place the AR art, but either which way, this emphasizes the importance of taking the user experience into consideration when designing AR exhibitions. Shade is important, both for the visitors’ physical comfort and for the practicality of visitors actually being able to properly see and experience the AR art work through their screens. With the size of the Royal Botanical Gardens there is more than enough options to place these 13 pieces of artwork in a way that keeps visitors out of the full sun and allows them to better see and experience the AR art on their screens.

Girl holding a tablet, demonstrating the difficulty seeing the AR art in the sun.

While certainly the audio experience would have been much better had we been able to use our headphones, as people do experience exhibits like this on a shared device, I think it would be beneficial for the audio portion of the experience to be added at a higher volume to better allow for visitors sharing a device to be able to hear the experience.

I appreciated that the artist shared a write up on their art within the app, but to spend time reading this, it takes you away from viewing and experiencing the AR art. It would be great, if visitors have the additional option to play an audio recording of the artist sharing their thoughts with visitors on the piece of art.

Finally, with the loaned tablets, it would be beneficial if the individual gardens made it possible for visitors to email themselves the copies of any photos they shot with the AR art.

Takeaways From This Extended Reality Experience

When easily visible and heard, the Seeing the Invisible Art Exhibition creates a glimpse into a hidden world that encourages thought and exploration through a new lens and perspective. Thats exciting, and demonstrated what excites me about sharing stories with augmented reality.

However, if the app isn’t working on people’s devices or the AR is difficult to see on people’s screens, then people become frustrated, rather than excited by the experience. This is why user design and testing is so very important. This should be thought of both in terms of the technology and visitors’ interactions with the physical environment in which they are experiencing the AR art. In the case of the RBG, the gardens and not trampling plants seems to have been taken into considerations, but not visitors’ physical comfort from the sun or thought of how that sun would impact user’s experience. This art exhibition was almost in its entirety in direct sun with no to very little shade.

A girl in another full sun AR art exhibit spot at the Royal Botanical Gardens.

User testing should be inclusive of the loaner tablets for visitor use, made available onsite. This means, if adding protective screen covers to the tablets, making sure those protective screen covers do not hinder visitors’ ability to enjoy the AR art.

It would also be advantageous to allow people using the borrowed tablets to be able to email themselves any pictures they took with the AR art, or if that is not possible, to be upfront with visitors about that from the start, so as not to leave them disappointed at the end of the experience.

Summary of Takeaways:

  • The technology exists to create a GPS triggered augmented reality app that works on people’s devices without overloading a device’s data.
  • The Seeing the Invisible AR App takes up 2.1 GB of space on my smartphone.
  • The Seeing the Invisible AR App was developed by Khora ApS, a virtual reality and augmented reality production studio in Copenhagen.
  • When the AR app is working, the augmented reality art exhibition creates wonder for visitors, gets visitors exploring and interacting in a new way, and has visitors engaging with the AR art.
  • By making the AR art exhibition viewable through smartphones and tablets (with loaner tablets available for use), it is accessible to all ages and most abilities.
  • User testing is important on the site of the experience, thinking about user comfort at different times of year.
  • Avoid creating GPS triggered AR screen experiences in direct sun.
  • Make sure protective screen covers on loaner tablet do not limit the ability to see the augmented reality.
  • Think about user safety and comfort when placing GPS triggered AR experiences.
  • Encourage visitors to bring headphones, while ensuring sound is available at a volume that those without headphones or sharing a device can hear.
  • People enjoy interacting with AR art.
  • Make sure people can email themselves any pictures they took with the AR art on loaned tablets.

Future Building

In reflecting upon the Seeing the Invisible AR Art Exhibition, in addition to placing the augmented reality art in spaces that people can enjoy the art in the shade and to be better able to hear the accompanying sounds when experiencing through a shared device, I’d also love to further encourage the engagement that people enjoyed with the art. Some possible ideas for that:

  • I’ve recently seen how the Relive App makes it easy to create videos from a walk by recording the map and editing in the photos and videos you take at different stops. Should this be possible within this AR app, it could create fun keepsakes from the AR experience, that also become valuable crowdsourced storytelling for the exhibit. An example from the travels of a fellow member of Vancouver’s tech community:
  • It would be great to be able to play the artist’s description of their work as an audio reading, complete with some prompts to encourage people to engage with their art in different ways.
  • I loved how Ori Gersht’s Forget Me Not was responsive to visitor’s movements, it would be exciting to have more pieces that respond to the location of the holder of a device.
Ori Gersht's Forget Me Not AR art flower arrangement.
Exploding vase of flowers, as a part of Ori Gersht's Forget Me Not responsive AR art.

The educator in me has also begun to design, in my head, a scavenger hunt that encourages visitors to collect experiences and different perspectives with each piece of art by encouraging visitors to take on some sort of challenge or unravel some sort of mystery with each piece of art. Being able to create a mapped storytelling account of such a scavenger hunt, in a format like the Relive App creates, would be a great way to further share the discoveries from such a scavenger hunt.

The storyteller in me would also love to learn more about the making of the art, like with this talk on Ori Gersht’s and Timur Si-Qin’s AR art pieces.

What’s Your Take on this Augmented Reality Experience?

I’d love to hear your thoughts in comments below on what you think would make this a richer and more accessible AR experience, and if you have technical solutions for making this a more immersive and user friendly experience.

One visitor using an umbrella to improve his AR art experience at the Seeing the Invisible Art Exhibition at the RBG, with all the sun.
One visitor using a black umbrella to improve his AR art experience in the sun.

References

Conklin, M., [Tucson Botanical Gardens], Maor, H., & Haring, T. (2022, December 1). A Talk with the Curators of Seeing The Invisible [Video]. YouTube. https://www.youtube.com/watch?v=1BXqc0g8D_c

Horwitz, L. (2023, June 12). phygital. Customer Experience. https://www.techtarget.com/searchcustomerexperience/definition/phygital

Jerusalem Botanical Garden. (2021, November 14). אחת העבודות בתערוכת האמנות, תפתיע אתכם במיוחד. . .. Jerusalem Botanical Garden’s Facebook Page. https://www.facebook.com/Jerusalem.Botanical.Gardens/videos/372403381331281

Rendell, H., Gertler, C., Katri, M., Maor, H., & Haring, T. (2021). SEEING THE INVISIBLE. SEEING THE INVISIBLE. https://seeingtheinvisible.art/

Royal Botanic Garden Edinburgh, Maor, H., Gersht, O., & Si-Qin, T. (2022, May 24). Seeing the Invisible  In Conversation with the Artists [Video]. YouTube. https://www.youtube.com/watch?v=3_XD4nmV-58

Royal Botanical Gardens. (2023, March 8). Seeing the Invisible – Royal Botanical Gardens. https://www.rbg.ca/things-to-do/art-in-the-gardens/seeing-the-invisible

Shamir, R., [America-Israel Friendship League], Gertler, C., Rendell, H., Maor, H., Haring, T., Rominiecki, J., & Firestone, W. (2021, November 8). Seeing the Invisible: Augmented Reality art [Video]. YouTube. https://www.youtube.com/watch?v=TIuxE1oZKtY

Parco Archeologico di Tremona-Castello AR Experience – an Extended Reality Case Study

by 8 Comments

Welcome to our new series exploring extended reality case studies, in a quest to develop richer and more accessible Extended Reality (XR) experiences. In this first case study, we delve into Parco Archeologico di Tremona-Castello’s (Tremona Castello Archaeological Park) Augmented Reality (AR) Experience and speak with Nadia Lupi, one of the visionaries behind the project.

Organization: 

Parco Archeologico di Tremona-Castello (Tremona Castello Archaeological Park)

Project:

Parco Archeologico di Tremona-Castello AR Experience

Type of Experience:

Augmented Reality Walking Tour

Device for Experiencing:

Augmented Reality Glasses paired with a guided audio tour

Link to Project:

https://www.parco-archeologico.ch/en/visita/la-realta-virtuale.html

Interview / Partook in the Experience Date:

April 30, 2019

Interview With:

Nadia Lupi, Director of Mendrisiotto Turismo

Country:

Switzerland

Delving Into the Tremona Castello Archaeological Park AR Experience & an Interview with Nadia Lupi

The Project : Parco Archeologico di Tremona-Castello AR Experience

Parco Archeologico di Tremona-Castello

Sitting on top of the hillside over looking the village of Tremona in the Mendrisiotto District of the Canton of Ticino, Switzerland, local archaeologist Alfio Martinelli, discovered the archaeological remains of an ancient medieval village sharing a history of the area for 6,000 years from prehistory to the 14th century. As Alfio and other archaeologists and volunteers began to uncover this village, many questions arose as they began the puzzle together the stories from the past. Among the curiosities of Parco Archeologico di Tremona-Castello are why they chose to build there, with no apparent water source on site.

Who lived here? How did they spend their daily lives? Why did they suddenly abandon this village in the 14th century? These are all questions that Alfio, fellow archaeologists, and volunteers excavating Parco Archeologico di Tremona-Castello have been asking, as they piece together pictures and stories of daily life in this ancient village when it was thriving.

It is believed that this rural village was inhabited by artisans and farmers from dark ages until the 14th century, who travelled widely and traded their goods. The people there made widespread use of money and owned beautifully handcrafted ornamental items. The site was likely established for the village due to its strategic position, which allowed its inhabitants to control the communication routes from Como and Varese to Riva San Vitale, Monte Ceneri and the central Alps. Excavations have brought to light a labyrinthine and well-fortified settlement that offered its inhabitants refuge from Barbarian invasions and the skirmishes between Como and Milan.

Why Augmented Reality?

These are stories that Alfio and others excavating the site wanted to share, along with Mendrisiotto Turismo. Initially Alfio and Nadia Lupi, the Director of Mendrisiotto Turismo, envisioned turning Parco Archeologico di Tremona-Castello into a open air, living history museum.

They did not get approval for an open air, living history museum, however, so had to think of alternative ways of helping others to envision life in Parco Archeologico di Tremona-Castello in medieval times. Thanks to 3D Artist and Animator Elia Marcacci volunteering on the excavation of Parco Archeologico di Tremona-Castello, Alfio and Nadia got the idea to have Elia create 3D animations of life in Parco Archeologico di Tremona-Castello in medieval times, which gave them the idea to create an augmented reality experience that would allow visitors to Parco Archeologico di Tremona-Castello to envision the site in medieval times, while still seeing the present day excavation.

The Augmented Reality Experience

In creating this augmented reality experience, Alfio, Nadia and Elia worked with the University of Applied Sciences and Arts of Southern Switzerland (SUPSI) to help shape and technically guide this experience.

They wanted to create an experience that would allow visitors to envision and experience life in the ancient village, while still seeing the excavated foundations of the ancient walls on the site today.

Initially Nadia had imagined this being experienced in AR through a smartphone or a tablet, with ghostly walls of the village and ghostly figures of villagers wandering about around you, going about their daily work and life.

While they do not have that experience yet, they do have an 10 detailed 3D scenes into the past, beautifully illustrated by Elia, including places where you can visualise the ancient village all around you.

The Medieval Village that once stood at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.
The Medieval Village that once stood at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.

Bordering on Italy there was concern that if this experience were made accessible through smartphones and tablets, then visitors would run into issues with their devices accidentally switching to Italian telecommunication networks and resulting in additional charges. Thus the AR storytelling was created to be experienced through AR glasses instead.

To make this a richer experience, sharing audio storytelling about the ancient village and Parco Archeologico di Tremona-Castello, the AR glasses were paired with an audio guide that both shares stories on the 10-minute hike from the visitors centre to Parco Archeologico di Tremona-Castello and within Parco Archeologico di Tremona-Castello.

Due to the AR glasses and the audio guide, visitors can only partake in this experience when the visitor’s centre is open. You can get the address and hours of the visitor’s centre by clicking the button below, as well as details on renting the AR glasses and the audio guide.

Plan for an AR Walk through Parco Archeologico di Tremona-Castello

My Reflections on this Augmented Reality Experience

Part of my goal in exploring different extended reality experiences is to discover what excites me in these experiences and what I find challenging, in order to reflect on what could create even richer and more accessible extended reality experiences.

Here are my thoughts on Parco Archeologico di Tremona-Castello’s AR experience, from when I visited it in 2019.

The Extended Reality Magic

The concept video for Parco Archeologico di Tremona-Castello’s AR experience that Elia, Alfio and Nadia designed is beautiful and would make for an incredible experience for visitors, immersing them in two times / worlds simultaneously through ghostly echos from the past.

Elia’s 3D imagery is also well researched, well thought out, beautifully illustrated, and depicts incredible detail.

A household scene, as depicted by Elia Marcacci, in in the Medieval Village once at Parco Archeologico di Tremona-Castello.
A household scene, as depicted by Elia Marcacci, in in the Medieval Village once at Parco Archeologico di Tremona-Castello.

Based on memory, the audio storytelling was also well done, to transport visitors through time and help them to better imagine life in the ancient village at Parco Archeologico di Tremona-Castello.

Current Challenges (as of 2019 Experience)

As Nadia mentioned, the Parco Archeologico di Tremona-Castello’s AR experience is not yet as she envisioned it in their concept video. In 2019, rather than seeing an active ghostly village around you, you can visit a number of 3D graphic illustrations of life in the village that are generally static, aside from minor movement, like a wisp of smoke. While these 3D illustrations are well done, when you are expecting to walk into an active ghostly village, this is disappointing.

For me, however, the most problematic part of this AR experience is the AR glasses. They are uncomfortable, are difficult to see through, especially if you yourself wear glasses, and are difficult to see the 3D scenes through. As such, visitors don’t get to fully appreciate how well Elia’s illustrations have been crafted.

Erica wearing AR glasses at Tremona Castello Archaeological Park

While the audio storytelling is well designed, the audio storytelling device is problematic, as it has to be set in the visitor’s centre and once you start it, it cannot be paused or replayed without returning to the visitor’s centre to have it reset. This means you can’t take the time to enjoy and immerse yourself in the Parco Archeologico di Tremona-Castello’s AR experience at your own pace, but must stick to the pace of the device. This also means that on a hot, sunny day, you can’t take breaks from the tour under the shade of a tree, but must stay in the sun, until the tour reaches its completion.

The AR glasses and audio guide also limit when visitors can enjoy the experience to the hours of the Visitor’s Centre.

Takeaways From This Extended Reality Experience

Nadia’s initial description of what she envisioned for Parco Archeologico di Tremona-Castello’s AR experience is similar to what I’ve been envisioning for the natural and cultural history augmented reality experiences that I’ve been dreaming of. It is exactly the experience that I was hoping for, and I suspect that with advancements in motion capture suits and world building becoming more accessible through game design engines, like Unreal Engine and Unity, it is becoming easier to make visions like this a reality in the future, making me excited for a future visit to Parco Archeologico di Tremona-Castello to see how the AR experience has evolved. It would seem that Elia has already constructed and made it possible to explore the outdoor spaces of the ancient village that once stood at Parco Archeologico di Tremona-Castello through his 3D animations. Having said that, I think tackling this in pieces of starting with specific scenes, rather than a whole ghostly village all at once, makes this much more manageable logistically and financially, as well as allowing for the opportunity to test drive and learn from the initial scenes before designing more.

The AR glasses were very problematic for me, however, and create an accessibility challenge and I imagine a large expense in establishing this experience. I personally would design a version of this that visitors could enjoy through their smartphones and tablets. With advances in smartphone and tablet technology, the concerns for visitors’ devices switching to an Italian telecommunications network, resulting in unexpected cellular charges, could be resolved in the following ways:

  • Creating a GPS triggered app for the AR experience, that visitors download before leaving the visitor’s centre. Visitors can then turn on airplane mode on their devices, and start the app while using the visitor’s centre wifi. A number of travel audio guide apps work this way, including the popular GuideAlong audio tours.
  • Create a WebAR version of the AR experience, and install a mobile wifi router (like a starlink mobile) at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode.

Being able to enjoy this AR experience through a smartphone or tablet would make it accessible to more people, would make Elia’s 3D illustrations easier to view and enjoy, and would allow people to enjoy the experience at their own pace, pausing and replaying it, if they wish. In this way, the AR experience could even be made accessible outside of the visitor’s centre’s hours, if so desired.

It should be noted that one advantage to AR glasses over WebAR is that the augmented realities resolution is usually better with AR glasses, but if the augmented reality is difficult to see through the AR glasses, then it doesn’t really matter how good the resolution is.

Food storage in the Medieval Village once at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.
Food storage in the Medieval Village once at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.

Summary of Takeaways:

  • creating a ghostly ancient village that can be viewed with augmented reality with ghostly figures actively taking part in village life is where the magic in this experience is for me
  • creating a series of ghostly scenes from the village is a more manageable starting point, and allows for the opportunity to test drive and learn from the initial scenes before designing more
  • AR glasses are uncomfortable for some and difficult for some people to see the augmented reality scenes in
  • AR and audio storytelling is more accessible when experienced through a smartphone or a tablet
  • concerns of smartphones or tablets switching to an Italian telecommunications network, could be resolved by having the AR experience accessible through a GPS triggered app or available through WebAR if a mobile wifi router was available for use at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode
  • being able to enjoy this AR experience through a smartphones or tablet would allow people to immerse themselves in the experience at their own pace, pausing and replaying it, if they wish

Future Building

In reflecting upon Parco Archeologico di Tremona-Castello’s AR experience, in addition to the desire to have this experience accessible through a smartphone or table, allowing me to pause and replay the audio storytelling, and to see the ancient village in ghostly form with active ghostly villagers, I’d also love to be able to interact with those scenes in some way. Some possible ideas for that:

  • the ability to take pictures with the ghostly village and villagers, including selfies
  • being able to listen in on the conversations of ghostly villagers
  • being able to learn an activity from the ghostly villagers, perhaps as they teach a ghostly child or apprentice
  • being able to partake in a festive event, by dancing or signing along with the ghostly villagers

The educator in me is also envisioning creating the village in VR for use in school classrooms, or possibly VR (or AR) teaser experiences that would get school kids and other visitors prepared for a visit with a bit of a background of the time period that they will be transported to and some questions or things to look out for and think about on their visit to Parco Archeologico di Tremona-Castello.

Exploring Tremona-Castello Archaeological Park with AR glasses.

What’s Your Take on this Augmented Reality Experience?

I’d love to hear your thoughts in comments below on what you think would make this a richer and more accessible AR experience, and if you have technical solutions for making this a more immersive and user friendly experience.

References

Hargreave, E., [StoryToGo]. (2023b, December 15). Tremona Castello Archaeological Park AR Experience [Video]. YouTube. https://www.youtube.com/watch?v=6f8NZaeC_do

La storia. (n.d.). https://aram-ti.ch/il-villaggio/la-storia/

Marcacci Elia. (2014, January 6). Ein Tag in Tremona-Castello. Medienarchiv. https://medienarchiv.zhdk.ch/entries/63853132-1225-498b-ba50-c0b38d3a9eb9

Martinelli, A. (2008). Tremona Castello. Dal V millennio a.C. al XIII secolo d.C. All’Insegna del Giglio.

MendrisiottoTurismo. (2016a, May 25). Il villaggio di Tremona-Castello in 3D [Video]. YouTube. https://www.youtube.com/watch?v=4gzp_A9ADrQ

Mendrisiotto Turismo. (2016b, August 10). Parco archeologico di Tremona-Castello, oggi [Video]. YouTube. https://www.youtube.com/watch?v=ri7HMuhSrVQ

MendrisiottoTurismo. (2016c, August 29). Anno Domini – Rievocazione storica al Parco archeologio di Tremona-Castello [Video]. YouTube. https://www.youtube.com/watch?v=hkCdTiknS-4

MendrisiottoTurismo. (2019a, May 15). Parco Archeologico di Tremona Castello [Video]. YouTube. https://www.youtube.com/watch?v=qHGKHO4bZBI

MendrisiottoTurismo. (2019b, July 12). Hunting for mysteries in time [Video]. YouTube. https://www.youtube.com/watch?v=Ty9LxvWVsOw

Mendrisiotto Turismo. (2023, May 19). La Regione da scoprire – Il Parco archeologico di Tremona 2023 [Video]. YouTube. https://www.youtube.com/watch?v=6BbQJVkru30

Parco Archeologico di Tremona-Castello (n.d.). Ticino Tourism. https://www.parco-archeologico.ch/en/

Società editrice del Corriere del Ticino SA. (n.d.). Tremona – TicinoTopTen. Ticino Top Ten. https://www.ticinotopten.ch/en/experiences/tremona

Exploring XR Development with my Final Master of Educational Technology Directed Studies

by 4 Comments

Join me in exploring XR Development for my final Master of Educational Technology Directed Studies, and finding ways of making XR Development more manageable and accessible for other educators, students and creatives navigating XR and beginning to develop their own projects.

One of my main goals in embarking upon my Masters of Educational Technology was to explore newer-to-me forms of storytelling that could help to bring education to life. Over the past 7 years as I’ve worked my way through my Masters, being introduced to UBC’s Emerging Media Lab and the work they are doing there, having been approached by a board member from the Juno Beach Centre to explore newer ways of sharing stories from World War II that would connect with Canadian school children, travelling to Normandy to visit the museum and see how they are sharing stories there and identifying the gaps in their storytelling, and being shortlisted and interviewed for an interactive and immersive storytelling fellowship in Norway and exploring the stories I might wish to create for that fellowship, the idea of delving deeper into education and storytelling through XR – Extended Reality, teased at my synapses. I took an early course in VR in Education, attended XR symposiums and showcases, began researching and exploring different applications of XR in education and storytelling, collaborated on an open course in Immersive Experiences in Natural and Cultural History Education, developed an idea for an AR storytelling app to share natural and cultural history experiences, and am working my way through other people’s courses in XR, including the University of Michigan’s Extended Reality series of courses. While I am only now beginning to write about all of what I have learned here, the series of posts that follow come from research and a rabbit hole that I began to travel through 6-years-ago, and to date have included four talks at national and international conferences in education, technology, and storytelling, sharing my findings and explorations, and one talk to a Journalism Masters Class at the American University of Cairo. I have shared that last talk below as it shares many of the goals that I set out to accomplish with this directed studies.

Defining Immersive and Interactive Media

Before we get into where my interests and observations are taking me with my final directed studies for my Masters, I thought we’d explore the terminology around immersive media, experiential media, and extended reality, as there is ambiguity around the terminology, so by addressing this at the onset, I hope to at least make it clear as to how I am using the terminology.

An Immersive art exhibit entitled, Interactive Diorama - Rembrandt 1632.
Interactive Diorama—Rembrandt, 1632, The Anatomy Lesson of Dr. Nicolaes Tulp / Aalto University, School of Arts, Design and Architecture, Helsinki. Photographed by Tom Mesic.

Immersive and Interactive Media / Experiential Media

Immersive and Interactive Media are basically the current term for what was termed transmedia, referring to media that is immersive and interactive in nature, allowing those that are engaged with it to experience it in a way that allows them to step into the experience and / or become a part of the media and / or interact with the media, potentially having an impact on how the experience is shaped.

Examples of forms of immersive and interactive media include virtual reality, augmented reality, escape rooms, 360 video, choose your own adventure stories, alternate reality games … and the list goes on. Another name for this form of media is experiential media

Extended Reality (XR)

Project Lead Olivia How, demonstrating Shakespeare xR's touch table of Shakespeare’s First Folio to interested onlookers.

Extended Reality is a form of immersive / experiential media that takes users into a new reality or places virtual objects into a user’s real world, enhancing everyday life with technology. Basically making impossible sensorial experiences, possible. (Hargreave, 2021a)

Included within extended reality are virtual reality (VR), augmented reality (AR), and mixed reality (MR).

Virtual Reality (VR)

Virtual Reality is a computer-generated environment with scenes and objects that appear to be real, making the user feel they are immersed in their surroundings. (Iberdrola, n.d.) While this often is experienced through the use of a headset, a user could also walk into a virtual reality environment created by projectors and screens. This simulated environment and experience could be similar to or completely different from that of the real world in which a user is physically situated.

Examples of virtual reality include 360 video, when experienced through a headset, meeting in Mozilla Hubs, and the Immersive Van Gogh Exhibit.

A student experimenting with the Occulus Quest in the Emerging Media Lab at UBC.
A student experimenting in VR at the Emerging Media Lab at UBC.

Augmented Reality (AR)

Augmented Reality (AR) is an interactive experience where the real, physical world is digitally augmented. The augmentation is usually visual in current applications, but it can also be through other senses, such as auditory or haptic. (Wong, 2021)

Examples of augmented reality include Pokemon Go, the ReBlink exhibition at the AGO, and selfie masking filters in SnapChat.

Exploring Tremona-Castello Archaeological Park with AR glasses.
Exploring Tremona-Castello Archaeological Park with AR glasses.

Mixed Reality (MR)

There is a lot of debate around what constitutes Mixed Reality. I have always thought of it and defined it as allowing for real and virtual elements to interact with one another, and for users to interact with virtual elements in a similar way they would in the real world or at least in a similar way that they might interact with a touch screen. (Hargreave, 2021b)

Students observing the Holobrain at UBC.
Observing the Holobrain at UBC, care of Kerry Blackadar.

Others have defined mix reality (Speicher et al, 2019) as:

  • a synonym for augmented reality
  • a combination of AR and VR
  • a stronger version of AR
  • a type of collaboration
  • the idea of aligning physical virtual environments. 
  • the spectrum from the real environment without any augmentation to the virtual environment, which is completely synthesized computer-generated virtual content. 

This is further confused by some researchers only working within the theoretical and not experimenting with the tools in real time and some corporations attempting to use the term ‘mixed reality’ to market the next level advances in extended reality.

Due to all the debate around defining mixed reality, and based on one research paper, the unlikelihood of a consensus between the various experts in research and industry, I will likely avoid the term mixed reality and instead describe the characteristics of various extended reality applications, and possibly where they lie on the reality–virtuality continuum developed by Paul Milgram and colleagues in 1994.

Reality-Virtuality Continuum

The Reality-Virtuality Continuum as proposed by Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino in their 1994 paper, encompasses all possible variations and compositions of real and virtual objects.  They hypothesized that everything between real and the virtual is mixed reality, and that mixed reality makes up both augmented reality and augment virtuality. (Milgram et al, 1994) Basically suggesting that mixed reality is the blending of the physical and the virtual world with varying degrees of augmentation. 

The problem with the RV Continuum in its current form is that it only takes into account visual, not sound, olfactory or haptics senses. (Speicher et al, 2019) It ultimately needs to be redeviced with how technologies are evolving.

Augmented Virtuality

Augmented Virtuality refers to perceiving mostly virtual content, while still seeing some real world content. (Nebeling, 2022)

My Interests and Explorations in Extended Reality

Virtual Reality

I have long been fascinated with extended reality, ever since I was first introduced to the concept of virtual reality in an episode of Murder She Wrote in 1993. It excited me to imagine how extended reality could be used to create immersive story experience and bring worlds to life that are created in one’s imagination. The problem for me though was that whenever I tried it, which I did over the years whenever an opportunity present itself, was that it was too much sensory overload for me, causing mr to feel motion sick (which I now know is cybersickness) and sparked migraines. (Weech et al, 2019)

Jessica Fletcher in a VR headset as she solves a 1993 Murder She Wrote mystery.
Jessica Fletcher in a VR headset as she solves a 1993 Murder She Wrote mystery.

Persistence pays off though, as in 2018 I finally had a VR experience that did not spark cybersickness or a migraine. The experience in question was a virtual plank walk designed to help people overcome their fear of heights. (Basbasse et al, 2023) In exploring the features that made the difference in this experience for me and talking to one of the designers of the experience, I believe the quality of the filming / graphics made the main difference in this experience, as well as perhaps the experience being designed to limit your time in VR. This also got me inquiring from the designer if VR was being used in other ways in the medical world, and if perhaps it was being used with pain management. My interest? By this point I’d been living with chronic pain following two car accidents for 5 years, and was curious if perhaps there were ways this technology could be used to help people better manage their pain. This sparked me to begin explore different medical avenues and uses to VR, as I began to contemplate what might be useful to me and to the community of chronic pain and fatigue sufferers that I now belonged to, many of whom are predominantly home bound.

VR Plank Walk
Epionia Therapeutic VR
Therapeutic VR for Pain Reduction

Augmented Reality

I have always loved and been drawn to educational experiences that incorporate storytelling and encourage the imagination. This was developed in me as a parks naturalist and is part of how I approached science education, and encourage kids to think, imagine, experience and explore.

In approaching work and storytelling, post the car accidents that very much changed my life, how I work, what I can handle and in some ways who I am, I have been increasingly drawn back towards natural and cultural history in my storytelling, as nature soothes my symptoms. When I travel I also love learning and love to envelop myself in the story of a space, but thanks to my post accidents realities I cannot always handle busy museums with noise, lights, and other sensory triggers. This got me imagining, first with my fellowship interview in Norway and later as I explored the Juno Beach Centre in Normandy, if I could create mobile storytelling experiences outside of the museums in places of natural and cultural history, where people could learn and enjoy the stories of the place as they explored outdoors. In both cases, as they were looking to me to imagine newer ways of bringing such stories to life, I began to imagine this with augmented reality as experienced through our mobile devices. This made me smile, as not only did this idea hold the potential for learning and storytelling, but also some of the fun, whimsy and magic that augmented reality brings with it. This is an idea that I further began to sketch out on my travels and in my ETEC 522 Course – Ventures in Educational Technology, and that I am working towards making a reality.

My Goals with this Directed Studies into Extended Reality

I have a tendency to dream big and my dyslexic brain sees connections and expanding and overlapping storyworlds. Usually this means that I do not have the money and often all the knowledge for orchestrating all that I imagine at the get go. My dyslexia has taught me to problem solve though, and life and an imagination and brain that don’t let go of ideas have taught me patience and to play the long game, approaching my ideas in digestible pieces. Thats what I have been doing with this directed studies. I essentially started the work on it after the first year into my Masters, slowly taking advantage of research experiences as they became available, participating in XR symposiums and showcases when they present themselves, taking formal and informal courses in XR, accepting opportunities to speak on what I am working towards with XR and learning along the way, and beginning to flush out my ideas and pitch them to funding bodies.

For this directed studies, my goals are to:

  • Write up the VR case studies that are helping me to identify the strengths and challenges in VR for patient communities.
  • Write up the AR case studies that are helping me to identify the strengths and challenges in AR for natural and history education.
  • Begin to develop and further experiment with VR and AR development.
  • Sharing what I have learned to help make it more manageable and accessible for other educators, students and creatives navigating XR and begin to develop their own projects.

A Beginning

While in some ways I see this directed studies as an end to my researching and a beginning to my developing, there will always be more research to be done in seeing how others are developing and applying the XR, especially as the technology is still developing and despite its decades of history, still very much in its early days for development, application and adoption.

Development-wise, waiting on gear and difficulties with my health and post concussion syndrome, have very much slowed down my intended goals with development and where I’d like to be at this stage, in terms of developing my own XR applications in VR for patients and AR for natural and cultural history. However, as my post accidents self has learned, pushing through when my health issues becomes problematic, only making things worse and setting me back further. Meaning that I have had to be patient with myself, have had to get my health back to a stable place (during a year that posed a lot of challenges with that), and have had to address the backlog of pressing deadlines from my health going sideways before I’ve been able to engage in development. I’ve put in the work to do that, and am now in a good and healthy space to embark on development. This is much later than I would have liked, but that is not something I can change with my newer disability realities post the car accidents. As such, my initial development goals for this directed studies will be smaller for now, and I will continue working on the bigger goals, post the directed studies and will continue to write those up as I do.

Erica wearing AR glasses at Tremona Castello Archaeological Park

This also means that for the Guide to XR Development that I will be developing to make XR Development more manageable and accessible for other educators, students and creatives, this will similarly take shape in stages and over time, some coming sooner with my directed studies and some continuing over time, as I further delve into XR development and experiments of my own.

Thank you for joining me on this journey into XR Development, and being patient with me as I take the time to do this in a way that is healthy for me with my health and disability realities. Hopefully my journey as the tortoise on this road will inspire others to explore and embark on such a journey of their own, no matter at what speed and working with whatever challenges your life and situation might pose along the way.

References

El Basbasse, Y., Packheiser, J., Peterburs, J., Maymon, C., Güntürkün, O., Grimshaw, G., & Ocklenburg, S. (2023). Walk the plank! Using mobile electroencephalography to investigate emotional lateralization of immersive fear in virtual reality. Royal Society open science, 10(5), 221239. https://doi.org/10.1098/rsos.221239

Hargreave, E. (2021a, November 15). An introduction to immersive experiences in natural & cultural history education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/intro/

Hargreave, E. (2021b, November 15). Mixed Reality (MR) in Natural & Cultural History Education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/mr-in-natural-cultural-history-education/

Hargreave, E., [Stories, EdTech & Digital Media with Ahimsa Media]. (2020, November 29). Story Steppers – an AR storytelling app to the natural & cultural history of a place [Video]. YouTube. https://www.youtube.com/watch?v=2xeWjBWYcAI

Hargreave, E., [StoryToGo]. (2023, December 10). Exploring Immersive Media to Develop Creative & Engaging Stories with Master of Journalism Students [Video]. YouTube. https://www.youtube.com/watch?v=Ff1cMMiH5cc

Iberdrola. (n.d.). Virtual Reality: another world within sight. Retrieved December 7, 2023, from https://www.iberdrola.com/innovation/virtual-reality

P. Milgram, H. Takemura, A. Utsumi, F. Kishino: Augmented Reality: A class of displays on the reality-virtuality continuum. In Proceedings of Telemanipulator and Telepresence Technologies, 1994.

Nebeling, M. (2022, June 2). Intro to AR/VR/MR/XR: Technologies, applications & issues. Coursera. https://www.coursera.org/learn/intro-augmented-virtual-mixed-extended-reality-technologies-applications-issues

M. Speicher, B.D. Hall, M. Nebeling: What is Mixed Reality? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 4-9, 2019.

Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Frontiers in psychology, 10, 158. https://doi.org/10.3389/fpsyg.2019.00158

Wong, Y. (2021, November 15). Augmented Reality (AR) in Natural & Cultural History Education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/ar/

You Need More Than an Elevator Pitch

by 2 Comments

With any position you hold, you most likely have a short description of it prepared for when anyone asks about what you do, commonly known as an elevator pitch. For more well-known positions, this may be enough for someone to understand your role but what happens if your field of work is new or more complex? Having a short response is necessary to have on hand but you also need to be prepared to answer follow-up questions and in some cases, defend your profession.

Art Therapy as an Example

Art therapy is a growing field and many people have yet to hear about it. Most art therapists will find themselves having to explain and defend their work to potential employers and people who are simply curious. With many clinics and organizations not yet utilizing art therapy, it is hard for art therapists to find jobs outside of private practice. Some art therapists have taken to reaching out to those sites in the hopes that they might consider adding an art therapist to their team. In this case, much more than an elevator pitch is needed.

Egyptian women painting with water colours, on the banks of Heisa Island Aswan, as a part of an art therapy session.
Art therapy in Aswan, as photographed by Rowan El Shimi (via Flickr).

Representing Yourself

When I say that art therapy combines models of counselling with the creative process to facilitate healing and growth, that is only a snapshot of a complex profession that holds entirely unique experiences for each client. That statement does not contain the nuances and different processes that may be involved during an art therapy session.

Illustrating anxiety on a colourful mind map.
Illustrating anxiety as photographed by Anima Visual via UnSplash.

When it comes to representing yourself you need to be able to answer questions efficiently and professionally. Questions you might ask yourself when preparing might be:

  1. What is my definition of my profession?
  2. What is my approach and how adaptable is it?
  3. What does a session typically look like? What are the applications of my profession?
  4. What population do I work with? Think ages, genders, cultures, socio-economic status, specific mental or physical conditions, etc.
  5. What is my intake process? (new client procedure)
  6. How do you know when a therapeutic/professional relationship should end?
  7. What does current research say about my profession?

Common questions people ask about art therapy are:

  1. How is art therapy different from art class?
  2. How is art therapy different from “art therapy” colouring books?
  3. What is the difference between counselling and art therapy? What are the benefits?
  4. How can art therapy work for me?
    • A question like this is the most important one to pay attention to when trying to promote yourself. It is easier for someone to see the usefulness of something when they can see it working for themselves.
    • This cannot be answered so simply without knowing a client’s needs and goals. Short answer: It would be determined and explained in the intake process and explored in the initial sessions as art therapy is applied differently for each client.
  5. If I bought an art therapy directive book, what would be the difference between completing it on my own versus working with an art therapist instead?
Rohingya children drawing pictures of what they witnessed in Burma.
Inside a UNICEF child friendly space, supported by UK aid, at Batukhali refugee camp in Bangladesh, art therapy helps Rohingya children recover from the trauma they have experienced. (Photographed by Anna Dubuis of DFID, care of Flickr.)

I will show how I would answer these questions later on in the article but please note that the answers will vary between different art therapists.

Be Prepared for the Unexpected

One night, I was out for dinner with some friends and we started talking about our jobs. One of them was familiar with the counselling process but had not heard of art therapy. At first, I gave my elevator pitch but they were curious to know more. As the conversation went on, it became clear that they did not see how art therapy could be effective. This caught me off guard as I was not ready to defend my profession at a casual dinner. I could feel the knowledge I needed retreating into the deep corners of my brain as the pressure set in with everyone waiting for my answers.

My friend’s questions revolved more around how art therapy could work for them. They also had a hard time understanding that an art therapy session is not the same as attending an art class or having someone supervise you while you make art. I talked about my approach to art therapy and how each art therapist ascribes to different psychological theories so the process would vary and also be tailored to the client’s needs. I also explained that similar to counselling, art therapy does not work for everyone. I did my best during this conversation, however, it ended with them still having a shaky belief in the effectiveness of art therapy.

After this conversation, I reflected on the questions and concerns that were raised so I could prepare myself for any similar situations in the future. Since I was caught off guard and didn’t have quick answers, I fear that contributed to my friend not taking myself or art therapy seriously. I honestly felt embarrassed that I had such a hard time since it made me feel that my profession was seen as a joke by someone close to me. Queue the imposter syndrome. I knew that what I have done so far with previous clients has been effective and made lasting impacts so I was able to find comfort in that. It is also near impossible to memorize every benefit and application of art therapy.

I want to take this time to acknowledge that in some cases, people have already made up their minds and there is not much or anything you can say to change it. Try not to be hard on yourself for not being able to gain someone’s support. Especially when you are unexpectedly on the defence.

Representing Myself as an Art Therapist

Below are my answers to the questions mentioned above. They will change as I progress through my career as I am a newcomer to this field. Again, the answers to these questions will vary between art therapists and it is common for clients to ‘shop around’ until they find an art therapist or counsellor that they connect and align with.

  • What is my definition of art therapy?
    • Art therapy is a mental health profession that combines models of counselling with the creative process to promote healing and growth. Through the use of art materials, personal exploration can lead to discoveries, understandings, and resolutions. No previous art experience is necessary!
    • Follow up: Sometimes words can be limiting in their definitions and expressing through art can expand those boundaries. It can transcend the spoken word and serve as a non-verbal means of expression.
  • What is my approach and how adaptable is it?
    • I take an anti-oppressive, person-centred and postmodern approach. Sometimes I will provide an art directive and sometimes I will leave the session completely open for you to create whatever needs to be expressed, also known as an open studio approach. I will also work with you to identify what is positive and functional in your life so that we can nourish what keeps you well.
    • I believe that you have the ability to bring the change you seek. You are the expert on your life and you have the power to take the session wherever you want. We will work together to find how much structure you require on your healing/growth journey.
    • An art therapy session with me is tailored uniquely for each client. I recognize that what works for one person, may not work for the next and that could include utilizing a different theory and therapeutic approach.
  • What does a session typically look like?
    • This varies for each client but it typically starts with a check-in where we discuss anything the client wants to bring up since the last session and if there is something specific they wanted to address that day. This is then followed by the art making and then of course a discussion of that art and the process. For some, there is no discussion after the art if they feel that is enough for them and it aligns with the goals that have been set. Some also talk during the creation process.
Students creating collages from magazine clippings in an art therapy class.
A Converse University student lead art therapy class. (Care of Flickr.)
  • What population do I work with?
    • While my practicum experience was primarily with elementary school-aged children and some seniors, I plan on working with all ages in the future. This may change as I gain experience and move through life.
  • What is my intake process?
    • This question is unique to each therapist and depends on multiple factors. Some may choose to offer a short phone or video call consultation to conduct the intake. Some may choose to have an entirely self-serve digital intake process and then discuss it during the first session. Since I am still a student, I don’t have a personal answer to this question yet.
  • How do I know when a therapeutic relationship should end?
    • Ending a client/therapist relationship happens uniquely for each client. Sometimes the end date is set during the initial sessions and sometimes the ending is based on set goals. It is the art therapists job to make note on when benchmarks are being met or not.
    • If progress does not seem to be happening, the treatment plan should be revisited and changed in the hope that it might be more effective. In some cases, a client may need to be referred to someone else.
    • If art therapy is not effective with one art therapist, it does not mean the client should give up on art therapy. As I have mentioned before, everyone has a unique approach with different perspectives. Compatibility is important.
  • What is the difference between counselling and art therapy?
    • While counselling and art therapy share similarities, art therapy can provide a non-threatening alternative to traditional talk-based therapy. Themes and topics are explored through art making which can be less threatening and confrontational for some. While talking can be a part of an art therapy session, a session can be done in complete silence and still be effective.
    • Art therapy can also address physical concerns and improve self confidence by increasing artistic abilities.
    • Some people see both a counsellor and an art therapist to get support in multiple areas.
  • What does current research say about my profession?
    • The short answer here is that research shows art therapy to be effective on a similar level to counselling but existing studies are not capable of showing the true effectiveness of art therapy due to how they were conducted (research limitations).
  • How is art therapy different from an art class?
    • Hopefully after explaining the points above, one would understand that difference but to elaborate further, art therapists do not teach clients technique or evaluate the quality of the art. The focus is on the process and the content, not the quality. A safe space is created for clients to not feel intimidated to meet certain standards.
  • How is art therapy different from ‘art therapy’ colouring books
    • While colouring books can feel therapeutic and serve as stress diffusers, they are not the same as working with a trained professional toward growth and change. They can be a tool during the therapeutic journey but the process requires much more. This answer would be the same if you were to purchase a book full of art therapy directives.
Soldiers exploring clay in the Resiliency through Art program at the Vicenza Art Center.
Soldiers exploring clay in the Resiliency through Art program at the Vicenza Art Center. (Image care of Flickr.)

You may not ever need to get this in-depth during a regular conversation but it helps to be prepared. Throughout my conversations with people, I’ve noticed that the main issue people struggle to get over is that art therapy is a form of psychotherapy and not an art class.

Pay attention to what people ask you when you are first getting into your field of work. Take note of the most common questions and misconceptions so you can be prepared with answers in the future. If you have any other business tips or stories about defending your profession, feel free to drop them in the comments!

Art Prompt: Strengths Fan

Paper fan, decorated with your words of strength.

In this summer heat, you need a fan to keep you cool! What better way to beat the heat than fanning yourself with your own strengths that you have identified about yourself? Okay, air conditioning is pretty cool too.

Materials Needed

  • paper (8.5 x 11)
  • stapler or paperclips
  • pens, coloured pencils, markers, etc

Instructions

  • Fold the piece of paper alternating directions until you have an accordion like piece of paper.
  • On each fold, think of and write a strength about yourself. Think about things that you are good at or important parts of your personality.
  • Decorate the fan in any way you please
  • Staple or paperclip the bottom to keep it together and you are ready to fan away

Reflections

This prompt is meant to help you think positively about yourself and identify how strong you are in various areas. Sometimes, the fan is left blank in some spots and that is okay. You may find yourself to be comfortable with the amount of strengths listed or you may use the blank spaces as inspiration to one day fill them with more strengths.

Bjorn Yearwood fanning himself with his strengths fan.

As usual, completing this is not equivalent to attending a session with an art therapist but is meant to give you a glimpse into what might be explored.

If you feel so inspired, share a comment below and share your strengths, tips and / or queries in sharing your elevator pitch. Thank you for reading!

Lessons Learned From Being a Clown and Becoming a Better Human Being

by 1 Comment

I began my clown journey knowing absolutely nothing.  I am pleased to say that having nothing in your head remains one of the key successes to being a clown.  Of course, when I started, I had no idea how useful not knowing anything was. In fact, it rocked me to my core for not being “good enough” when I initially started.

Jennifer Rose Garcia and fellow students at Ecole Philippe Gaulier, in full clown regalia.
A couple of my clown friends and I clowning around at clown school.

I came to clown school at Ecole Philippe Gaulier with a wealth of experience as a performer and lover of the arts.  Many years of my youth consisted of training in areas of theatrical acting, musical theatre, show choirs, dance, and acting for film and television work.  One area I did lack in was the incredible theatre sport that is improvisation.  I was told by one of my former acting instructors that clown was “improv on fire”.  I knew that if I were to venture into clown work that my improvisation skills needed to be sharpened.  

I took a couple of local improv classes, so I understood the basic requirements of improvisation, but it was never my go-to form of performance.  I am definitely an over-thinker to a fault, and love to make plans – but being a teacher as my day job allowed me to improvise (or as others put it be “flexible”) in order to maintain engagement.  In a way, this kind of improvising is useful to a clown – as its main goal is to “win over” the audience.

I will say that my journey in France and introduction to clown was a rough one, but I have definitely learned a ton along the way.  The following list is a generalized summary of what I have gathered in the world of clown, and how it has re-informed my understanding of performance, connection and humanity.

Clown instructor Philippe Gaulier surrounded by students in clown regalia.
My reaction after being pinched in the face by the infamous clown instructor Philippe Gaulier in Étampes, France.
  1. You’re not as clever as you think you are.  If you come in with an idea you think is brilliant to your own standards and come out to the stage with confidence that you are brilliant, your idea will most likely make you flop*.  Instead, listen to your gut and let in others into your initial spark of an idea, and adjust accordingly.  You will be amazed with what you can accomplish when you show up as you are, and just embrace where you are emotionally and physically in the moment.
  2. Embrace failure.  It’s okay flop*, when you try something for the first time.  It’s the earnest effort that counts.  You get bonus points for trying again with that same earnest effort.  Embrace the flop – most times, your audience responds more genuinely to your reaction to your failure in a kind a way, and you end up winning over the audience anyway.
  3. Embrace change.  When you can feel the flop* arising from the audience, change what you are doing immediately – or embrace and accept the flop.
  4. Enjoy the glow when you’re in the show.  When something is going well on stage, and you are getting laughs, DO NOT change what you are doing.  Keep doing the same thing (saying the same things or doing similar gestures), because apparently what you are doing on stage is working and you’re winning over the audience!
  5. Eye contact and connection are key.  Don’t look up into the sky, or the ground, or behind your audience.  Do not be afraid to make eye contact and make connections with the audience.  It’s okay to engage even in a short conversation with them.
  6. Do your very best.  If a clown does not care, the audience does not care.  Care about your task, regardless how silly or tedious or challenging it may be.  When the clown tries their best, the audience will root for your success and love you regardless if you are successful or not.  A clown’s job is to “save the show” — do your best doing so!
  7. Do not hold a grudge.  Throwing blame or shame will blow up in your face.  Do not get angry at the audience for not responding positively to your performance because it will be very tough to win them back.  Instead use your flop* as an opportunity to mentally recoup, listen to what the audience is telling you through their body language and adjust what you are doing. 
  8. Start in a place of joy and curiosity.  When a clown begins a performance, they must be curious about the audience, task or activity – and they should have fun doing so!  This energy is contagious when spread genuinely, and the audience will be enticed and curious to what the clown will do next.  Be warry of those who start off grumpy – this energy is also contagious.
  9. When all else fails, follow your heart.  When a clown is floundering, and if reverting to previous performance tactics are not working and a flop is at about to arise, think about what makes your heart happy and follow that guttural instinct.  Sing, dance, talk about something you adore – whatever it is, allow it to bring the sparkle in your eye to let the audience in on the love you experience.  When you feel it, the audience feels it too.
  10. Exit when you’re prompted to.  When we’re killing a performance, we wish it goes on forever.  We want the audience to feel like putty in our hands and will want to earn that feeling again and again on stage.  Unfortunately, there will be a time where that moment ends.  It’s not necessarily a flop* or sign of defeat, but all great things come to a natural end, and we too, clown and humans alike, should embrace that finish line and know when it is time to move on.
Jennifer in a clown exercise with experienced clown Yuriko Ogino.
Me in my first week of Clown School at Ecole Philippe Gaulier.

These lessons learned in clown have not only allowed me to learn how to open up on stage, but have allowed me to open up in my personal life.  I feel like at our very core, all of us humans are clowns.  We live our lives seeking love and validation whether we like it or not.  We respond graciously to positive reinforcement and constantly adjust ourselves in response to others.  We listen, receive and adapt constantly, doing our very best to survive and to save the “show”.  The “show” which symbolizes our existence among those we cohabitate.  Like a clown, we yearn to connect, love and be accepted for who we are, flops and all. 

Clown Jennifer taking a bow in front of the Ecole Philippe Gaulier poster.
Me at the end of my Second Week of Clown School at Ecole Philippe Gaulier.


*flop: (of a performer or show) be completely unsuccessful; fail totally