StoryToGo

Exploring storytelling in its many forms in this digital age

Connect with our Community

Erica Hargreave

Submitting to the 2024 TMAC Awards on FilmFreeway

by Leave a Comment

We are excited to launch the 2024 TMAC Awards on FilmFreeway, an awards submission platform that should help to streamline our submissions and judging.

As the submission process will be new for the TMAC Awards, we have created the tutorial below to walk you through this process.

TMAC Awards on FilmFreeway

Guide to Entering the TMAC Awards on FilmFreeway

You will find the 2024 TMAC Awards on FilmFreeway at https://filmfreeway.com/TMACAwards.

In this Guide, we share instructions on:

  • Creating an Account on FilmFreeway
  • Adding Your Project for the:
    • Written Word Award Categories
    • Photography (aka Visual) Award Categories
    • All Media Forms or Best Emerging & Broadcast Media Award Categories
    • Self-Published Travel Website (including Blogs) Award Category
  • Submitting Your Projects to the TMAC Awards

Create an Account on FilmFreeway

To enter the 2024 TMAC Awards, you will need an account on FilmFreeway. This is free. Just click Sign Up on the Header Menu (or Log In, if you already have a FilmFreeway account).

Sign Up for the TMAC Awards on FilmFreeway

In signing up, be sure to select “I want to submit my work or get tickets to festivals.”

Select "I want to submit my work or get tickets to festivals."

Once you are signed into FilmFreeway, click on the downward arrow beside your avatar in the Header Menu and click on Account Settings.

Navigating to Account Settings on FilmFreeway

Scroll down to Currency and change your currency to Canadian dollars.

Change the Currency You Are Paying in on FilmFreeway

Add ‘Your Projects’

Before you enter an Awards Category, you need to ‘Add Your Projects’ to your FilmFreeway Portfolio. By ‘Projects’ this refers to the stories you wish to submit to the 2024 TMAC Awards.

Add A Project on FilmFreeway

Before starting, you will want to visit TMAC’s Website and review the 13 National Awards Categories to decide which categories you wish to enter.

Remember you can enter:

  • a maximum of 6 of your stories published between January 1 and December 31, 2024, unless otherwise stated;
  • a maximum of 2 stories in any given category; and
  • the same story cannot be submitted more than once (even if in a different category).

Please be sure to review the full list of TMAC 2024 Awards Submissions Guidelines.

Written Word Award Categories

For projects that you intend to enter into one of TMAC’s Written Word Awards Categories follow these steps:

  • Fill in the Project Information with:
    • Project Type: Script
    • Project Title: The name of the article, blog post, or book that you are submitting.
    • If your project is in French, click on the “My Project also has a non-English Title and Synopsis” box, and fill in the Project Title en Francais.
    • Add a link to your article, blog post, or book (optional).
TMAC Written Word Project Information on FilmFreeway
  • There is nothing you need to do with the Submitter’s Information, aside from making sure that it includes the same email as your login for TMAC’s website.

  • Under Credits, add the names of all the writers of the article, blog post, or book.
Add your project credits on FilmFreeway - TMAC Written Word Awards
  • Select ‘Other’ under Project Type on Specifications.
Project Type on FilmFreeway
  • Ignore the Past Awards Section and click Save Project.
  • Under Upload Script File, select ‘No’ for “Does your script file include a cover page?” and then click ‘Choose File’ to upload a PDF copy of your article, blog post, or book. This is a requirement for all TMAC Awards Written Word Category Submissions.
  • If anyone is having trouble creating a PDF of a web based article, try FireShot. It is free. A nod of thanks to Jennifer Bain for this suggestion.
Upload article on FilmFreeway
  • If you are submitting a book, then please mail two physical copies or PDFs for judging to TMAC PO Box 332, Brighton, ON, K0K 1H0.
  • If submitting a book with two or more Media Members listed as authors, contact Christine before submitting.
  • Congratulations! You have added everything needed for this Written Word Award Submission. Now click on ‘My Projects’ in the Header Menu to add your next story, or scroll down in this article to Submit Your Project to an Award.

Photography (aka Visual) Award Categories

For projects that you intend to enter into one of TMAC’s Photography (aka Visual) Awards Categories follow these steps:

  • Fill in the Project Information with:
    • Project Type: Photography / Design
    • Project Title: The name of the photograph or the article or blog post in which the photograph was published
    • Add a link to the article, blog post, or book that your article is published in (optional).
TMAC Photography Project Information on FilmFreeway
  • There is nothing you need to do with the Submitter’s Information, aside from making sure that it includes the same email as your login for TMAC’s website.
  • Under Credits, add your name as the photographer.
Photographer Credits on FilmFreeway
  • On Specifications fill in the Date the Photograph(s) was Published, rather than the Date the Photograph was Taken.
  • You don’t need to fill in anything else on Specifications, but you can if you wish to.
Photography Specifications on FilmFreeway
  • Ignore the Past Awards Section and click Save Project.
  • Upload Photographs. This is a requirement for all TMAC Awards Photography Category Submissions.
Upload your Photographs on FilmFreeway
  • Under Files & Attachments upload a PDF copy of the article, blog post, or book that your photograph is published in. This is a requirement for all TMAC Awards Photography Category Submissions.
  • If anyone is having trouble creating a PDF of a web based article, try FireShot. It is free. A nod of thanks to Jennifer Bain for this suggestion.
Add a File on FilmFreeway
  • Congratulations! You have added everything needed for this Photography Award Submission. Now click on ‘My Projects’ in the Header Menu to add your next story, or scroll down in this article to Submit Your Project to an Award.

All Media Forms or Best Emerging & Broadcast Media Award Categories

For projects that you intend to enter into one of TMAC’s All Media Forms or Best Emerging & Broadcast Media Awards Categories follow these steps:

  • Fill in the Project Information with:
    • Project Type: Select Film / Video or Script (for Written Projects) or Music (for Audio Projects) or Photography / Design or VR / XR / Immersive, depending on what best suits your project.
    • Project Title: The name of the story that you are submitting.
    • If your project is in French, click on the “My Project also has a non-English Title and Synopsis” box, and fill in the Project Title en Francais.
    • Add a link to your story (optional).
  • As you’ve already seen examples of the choices for Scripts and Photography, below you will see an example for Film / Video.
TMAC All Forms of Media Project Information on FilmFreeway
  • There is nothing you need to do with the Submitter’s Information, aside from making sure that it includes the same email as your login for TMAC’s website.
  • Under Credits, add the names of your project’s main creatives.
All Forms of Media Credits on FilmFreeway
  • Select whichever Project Type fits under Specifications.
  • On Specifications fill in the Publication Date instead of the Completion Date.
  • You don’t need to fill in anything else on Specifications, but you can if you wish to.
All Forms of Media Specifications on FilmFreeway
  • Ignore the Screening / Distribution Section and click Save Project.
  • You must upload a PDF, under Files & Attachments with a description of the project, form of media, and a link to it (if it is online). This is a requirement for all TMAC All Media Forms or Best Emerging & Broadcast Media Awards Category Submissions.
Add PDFs and Photos on FilmFreeway
  • Additionally, please include one of the following:
    • Under Add an Online Screener, upload a video under Upload.
    • Under Add an Online Screener, add a Link to your story. This can be a link to any form of media.
    • Under Upload Script File, upload a PDF of an article, blog post, book, or other form of media.
    • Under Upload Photographs, add any photos that you are submitting as your story.
  • If anyone is having trouble creating a PDF of a web based article, try FireShot. It is free. A nod of thanks to Jennifer Bain for this suggestion.
Add Your Video or Project Link on Film Freeway
  • Congratulations! You have added everything needed for this All Media Forms or Best Emerging & Broadcast Media Award Submission. Now click on ‘My Projects’ in the Header Menu to add your next story, or scroll down in this article to Submit Your Project to an Award.

Self-Published Travel Website (including Blogs) Award Category

For projects that you intend to enter into one of TMAC’s Self-Published Travel Website Award Category follow these steps:

  • Fill in the Project Information with:
    • Project Type: Script
    • Project Title: The name of the website that you are submitting.
    • If your project is in French, click on the “My Project also has a non-English Title and Synopsis” box, and fill in the Project Title en Francais.
    • Add a link to your website.
  • There is nothing you need to do with the Submitter’s Information, aside from making sure that it includes the same email as your login for TMAC’s website.
  • Under Credits, add your name as the website creator.
Add your project credits on FilmFreeway - TMAC Written Word Awards
  • Select ‘Other’ under Project Type on Specifications.
Project Type on FilmFreeway
  • Ignore the Past Awards Section and click Save Project.
  • Under Upload Script File, select ‘No’ for “Does your script file include a cover page?” and then click ‘Choose File’ to upload a PDF with the name of your website, a link to your website’s homepage, and a link to three articles that you personally wrote in 2024. This is a mandatory requirement for all TMAC Awards Self-Published Travel Website Category Submissions.
Upload article on FilmFreeway
  • Optional: Under Files & Attachments upload a PDF copy of each of the three articles or blog posts that you have written in 2024, that you wish judged as a part of this award.
  • If anyone is having trouble creating a PDF of a web based article, try FireShot. It is free. A nod of thanks to Jennifer Bain for this suggestion.
Add PDFs and Photos on FilmFreeway
  • Congratulations! You have added everything needed for this Self-Published Travel Website Award Submission. Now click on ‘My Projects’ in the Header Menu to add your next story, or scroll down in this article to Submit Your Project to an Award.

Submit Your Projects to the TMAC Awards

You are now ready to submit your projects to the TMAC Awards.

As a reminder you can enter:

  • a maximum of 6 of your stories published between January 1 and December 31, 2024, unless otherwise stated;
  • a maximum of 2 stories in any given category; and
  • the same story cannot be submitted more than once (even if in a different category).

Please be sure to review the full list of TMAC 2024 Awards Submissions Guideline.

You will find the entry forms for 13 TMAC National Awards Categories on:

TMAC Awards on FilmFreeway

Provide you use the following three waiver codes, your first three entries will be free:

  • TMAC2024Submission1
  • TMAC2024Submission2
  • TMAC2024Submission3

You can use each waiver code only once. Your 4th – 6th entries will cost $25 CA each + a service fee of $1.79 CA / entry.

Submitting will look like this:

Submitting to an Award on FilmFreeway in CAD

In order to use your waiver codes, submit your projects at the Checkout, one at a time, on your first three entries. This is what the Checkout will look like:

Adding Waiver Code at Checkout on FilmFreeway

…. and once your Waiver Code is applied ….

Successfully Added Waiver Code at Checkout on FilmFreeway

Should you have questions regarding submitting your entries through FilmFreeway, first review the instructions above to see if your question has already been answered. If it has not, then send an email to Christine or Erica.

Good luck in the awards!

2024 Soft Launch of the New Interledger Web Monetization Browser Extension, Relaunching the Web Monetization Standard Ecosystem

by Leave a Comment

Want to earn from your digital assets? Not a fan of ads, affiliate marketing, and data mining, or want to ad an additional way of earning from the digital media that you create? Then try experimenting with the Web Monetization Standard, which is relaunching its ecosystem with the Interledger Web Monetization Browser Extension.

What is the Web Monetization Standard?

Simply put, Web Monetization is a way that digital content creators can earn from streaming micropayments based on the amount of time that visitors spend on their digital content.

Interledger Web Monetization Extension
Image care of Coil

In technical terms, the Web Monetization Standard is a JavaScript browser API that allows for the creation of a payment stream from a website visitor to a website.

How Does Web Monetization Work?

By placing an Interledger enabled payment pointer for a digital wallet on the backend of a digital asset, anyone visiting that asset with the Interledger Web Monetization Browser installed and an active sum of money (‘grant’) to send, will send a flow of micropayments from their wallet to your payment pointer for the time spent on your digital asset.

Image care of the Artist Rescue Trust

Additionally, they can decide to support your further with a one time payment.

Try Web Monetization Out With Play Money

  • Sign up for a Play Money Wallet at https://wallet.interledger-test.dev
    • As a part of this, the sandbox has you fill in a KYC, as you would with any digital wallet. Be forewarned, it is annoying, as this is a pretend wallet though, you can put in pretend information. The only things that have to be real for it to work are your email, phone number and face scan.

*If you don’t wish to go through the KYC as it is frustrating and can be intimidating, send me an email at erica[at]storytogo.ca and I will set up a play money test wallet for you, so that you can get to the experimenting with none of the frustration.

  • Create a Payment Pointer
    • Once you’ve created your account, click on the existing EUR wallet or set up a new wallet in a currency of your choosing.
    • Inside the wallet of your choosing, click on ‘Deposit’. (as this is a play money account, add as much money as you’d like.)
    • Still inside the wallet of your choosing, click on ‘Add Payment Pointer’ and create a payment pointer.
Create a Payment Pointer in the Interledger Play Money Test Wallet

Send Money with the Interledger Web Monetization Extension

  • Download the Interledger Web Monetization Extension. As these are in soft Beta, we are tracking who has access and is experimenting with, so please ask if you would like access.
  • Add your Payment Pointer to the Interledger Web Monetization Extension and set the ‘budget’ amount you wish to spend for the month.
  • When prompted, click ‘Agree’ and ‘Accept’ to give the required permissions between the Interledger Web Monetization Extension and your Play Money Test Wallet.
  • Click back on the Interledger Web Monetization Extension and click on the gear icon in the top corner.
  • Here you will find options in the Interledger Web Monetization Extension to change your monthly budget and the rate at which you wish the micropayments to stream between your extension payment point and the payment pointers of the digital assets you are spending time on. This rate can be changed at any time, or paused, if on a site you do not wish to send funds to. The budget can be changed month or made reoccurring.
  • Visit a Web Monetized Website.
    • Find a list of Web Monetized Websites here. (Stay tuned – updating shortly.)
  • Look at your Interledger Web Monetization Browser Extension. It should show a green dot if the site is Web Monetized or a red dot if it is not. If you see a green dot, try sending a one time payment.

Earn Money with the Interledger Web Monetization Extension

  • Place an Interledger enabled payment pointer to your digital wallet in your website’s header or footer with this code:
<html>
  <head>
    <title>StoryToGo</title>
    <link rel="monetization" href="$ilp.interledger-test.dev/thankssantosh">
  </head>
</html>

OR

  • Add Coil’s Web Monetization Plugin (yes, the plugin still works, and I’ve been promised it will be further updated) to your WordPress site and add Interledger enabled payment pointer to your site via the plugin.
  • Add your Website to this editable Google document of Web Monetized Websites. (Stay tuned – updating shortly.)

Earn or Send Read Money with the Interledger Web Monetization Extension

  • Repeat the above steps with an Interledger enabled wallet. Current Interledger enabled wallets include:

*This section will be updated this week with links to instructions for each of these real money wallets above.

Web Monetization is in Soft Beta

With the relaunch of the Web Monetization Ecosystem with the Interledger Web Monetization Extension, Web Monetization is currently in soft beta. This means that there are limitations at present.

Usage Limitations

While the Web Monetization ecosystem is built back up, this presently means earning from Web Monetization currently only works on websites.

Stay tuned for the ability to earn in the future from:

  • video
  • podcasts
  • music
  • social media platforms
  • and more

… as the Web Monetization ecosystem grows.

Wallet Limitations

The wallets are also in soft beta with Web Monetization, which means at present:

  • you cannot send between different wallet types (ie. Fynbos US wallets can only send to Fynbos US wallets. A Fynbos US wallet cannot currently send to a GateHub US wallet.)
  • you cannot send between currencies (ie. Fynbos Canadian wallets can only send to Fynbos Canadian wallets. A Fynbos Canadian wallet cannot currently send to a Fynbos US wallet.)

These will change, but for now, so as not to lose out on earnings, add all your payment pointer to your website header or footer, like this:

<html>
  <head>
    <title>Naturally Ours</title>
    <link rel="monetization" href="https://fynbos.me/lori"> <link rel="monetization" href="https://fynbos.me/erica"> <link rel="monetization" href="https://ilp.gatehub.net/276288680/USD"> <link rel="monetization" href="https://ilp.gatehub.net/276288680/EUR"> <link rel="monetization" href="https://ilp.chimoney.com/90711674_1724964400719"> <link rel="monetization" href="https://ilp.rafiki.money/wm-receiving">
  </head>
</html>

Stay Tuned to StoryToGo.ca/web-monetization-standard/

Stay Tuned to StoryToGo.ca/web-monetization-standard/ for updates in the Web Monetization Ecosystem, including resources and examples of how Web Monetization is being applied to build sustainability.

Conceptualising an Extended Reality Open Education Guide

by 6 Comments

As we step into 2024 and I wrap up my ETEC 580 Directed Studies and Master of Educational Technology, I want to take a moment to conceptualise the Extended Reality Open Education Guide that I have begun to build; to give it shape, intention and create a road map for it, as well to seek feedback from you, our readers, as I continue to further my journey, experiments, and goals in extended reality (XR) beyond my Masters. As I previously wrote in Exploring XR Development with my Final Master of Educational Technology Directed Studies, the work I have done through the MET Program at UBC is a beginning to this work in XR development.

Here’s where I am at present in mapping out where I am going with this work:

XR Open Education Guide for StoryToGo

As I continue to develop an XR Open Education Guide on StoryToGo, much as I did with the Web Monetization Guide, there are 4 main areas that I will be focusing on, developing, and building upon over time:

  • XR Case Studies
  • XR Development Experiments and Process Blogging
  • XR Development Series of Mini-Courses
  • Addressing Accessibility in Extended Reality

XR Case Studies

The goal of the Extended Reality Case Studies is to explore different extended reality experiences and applications to discover what resonates with myself and my team and to identify the challenges. By writing these up as case studies, the hope is to create a resource for our team in developing extended reality experiences, and to share that resource more broadly with other independent creatives, educators and students wanting to explore extended reality. Secondarily, these XR Case Studies will also be used as resources and examples within the XR Development Series of Mini-Courses.

At present, these XR Case Studies are focused around augmented reality (AR) natural and cultural history experiences and virtual reality (VR) health and wellness experiences, as these are the areas that my team and I are currently developing extended reality projects. Over times, we will broaden these case studies to other areas of interest.

Extended Reality Case Studies

As my team’s and my current priority is on AR natural and cultural history experiences, the current case studies include:

I am also writing up a case study on the TaleBlazer AR Game at the Royal Botanical Garden’s Rose Garden at present.

Other XR Case Studies on the current ‘to write’ list, include:

  • Notre-Dame de Paris, The Augmented Exhibition
  • Plank Walk VR
  • Pain Distraction VR
  • Concussion Therapy VR

As the XR Case Studies grow in number on StoryToGo, we will create an Experiential Reality Library, like the Web Monetized Content Library that we’ve begun to create.

XR Development Experiments and Process Blogging

As an indie creative who likes experimenting with emerging ways of crafting stories, I find it helpful to experiment with the technology for building those stories. That way even if we contract a larger team on our builds, I know what I am talking about and know what is possible. Often as I dive into such experiments, I also learn that building stories in these newer ways with emerging technologies is not so daunting, cost prohibitive, and inaccessible as people think. Thus, as I create and experiment with extended reality development, using different extended reality development tools and platforms, I intend to create process blogs walking people through the ‘how to’ of my experiments creating with different technologies.

The purpose?

  • to create a running log / record of the process to pull from for the XR Development Series of Mini Courses;
  • to engage others around this process, and crowdsource ideas and solutions; and
  • inspire others to experiment with their own XR builds, and provide them with a resource to helping them do so.
Extended Reality Development Experiments and Process Blogging

Currently my goals for my own projects involve Unreal Engine and motion capture technology. This, however, is not the most accessible entry point for educators and students wanting to begin their own experiments with XR, both in terms of the learning curve and in terms of the expense of the technology needed for. As such, I decided for the purposes of creating a more accessible entry to extended reality for teachers and students, that I want to begin with some more accessible extended reality builds / development ideas, and then build towards the more involved pieces using UnReal Engine and Unity, as part of my long term, on-going work.

What this means, is that my initial AR build will be with TaleBlazer to create some early development options within the open Extended Reality Course that are designed for teachers and students, and then later build Units on developing with UnReal Engine and Unity, as well as other more accessible extended reality development solutions, like Stornaway.io, CoSpaces, Halo AR, polycam, MyWebAR, and 8th Wall. If you have other suggestions of XR development tools and platforms that my team and I should be experimenting with, please let me know about them in the comments on this post.

In terms of the extended reality projects that my team and I have been developing, they include:

  • WWII stories from my friend Manami Calvo (Saito)’s family from their experience as Canadians of Japanese ancestry, living on the West Coast in WWII
    • our plan for this is to create some initial experimental ‘art installations’ using Web AR, that are accessible via smartphones and tablets
    • we plan on creating these utilizing animated ‘ghostly figures’ created with Unreal Engine and a motion capture suit, along with historic photos and video, as well as recorded interviews
    • we will then use those art installations to go after proper funding to further development

  • health / wellness VR Experiences
    • our plan is to start with a kids yoga class in VR that can be used in hospitals, as well as at home
    • create in Unreal Engine with a motion capture suit
    • the yoga will be lead by Lori Yearwood (wearing the motion capture suit) and will involve her transforming to different plants and animals with different poses
    • the children will also be able to select the background world (from a variety of calming choices) for their yoga experience

XR Development Series of Mini-Courses

To make things as easy as possible for other indie creatives, educators and students to begin to explore and develop their own extended reality projects, I intend to take what I have learned through my extended reality research, case studies, experiments and process blogging, and build a series of Extended Reality Mini-Courses for people to use as a guide in creating their own extended reality projects. The goal here being to remove the fear of extended reality being a far reaching, inaccessible goal by building their understanding of extended reality, getting them experimenting with it and envisioning the possibilities, and sharing with them a variety of pathways to creating extended reality, including step-by-step guides.

Extended Reality Development Courses

In developing a series of Extended Reality Mini-Courses, I am exploring other Extended Reality Learning Resources to see what is currently available to those wishing to explore extended reality further, so as best to identify a different, unique approach to the subject and include any beneficial resources that are currently missing.

The Extended Reality Learning Resources that I am currently delving into, include:

Pre-Existing Extended Reality Courses

In approaching the extended reality course development for the StoryToGo Classroom site, I have come to the conclusion that XR development is more manageably approached as a series of mini-courses. Currently, I am thinking in terms of building the following three courses:

  • First Steps in Extended Reality Development – An Introduction
  • Extended Reality Development
  • Extended Reality User Testing and Refining

The talk I gave for the class at the American University in Cairo follows the layout for the first mini-course: 

My current breakdown for ‘First Steps in Extended Reality Development – An Introduction’ will include:

  • What Extended Reality Is
    • types of Extended Reality
    • applications of Extended Reality
    • delivery of Extended Reality and Devices

  • Extended Reality Case Studies
    • an informal introduction to user testing
    • creating mini case studies in user testing extended reality experiences, and recording stars and wishes from the experiences
  • Developing an Extended Reality Idea of Your Own
    • Why XR?
      • what type of XR?
      • is this the strongest choice?
      • how do you wish to deliver? to which devices? why?
    • Do you need to create all at once or can you create it in accessible steps?
    • Steps in creating
    • Creating a roadmap

Then the Extended Reality Development Course will be a continual work in progress, adding different Units, covering how to build XR with different extended reality development solutions, starting with TaleBlazer.

Extended Reality Mini Course to Develop for StoryToGo

If you have suggestions for the structure of or things I should be including in the XR Development Series of Mini-Courses, please share your thoughts in the comments.

Addressing Accessibility in Extended Reality

While not a standalone piece on my brainstorming document, for those of you who have begun to read our XR Case Studies, you may have recognized that accessibility in extended reality is central to my team’s and my explorations and development goals. This is very much driven by various disabilities and health and wellness issues that my team and I have experienced, but is also an aspect of extended reality design and user testing that has not been discussed and addressed as well as it should yet.

For now, my team and I intended to address Accessibility in Extended Reality throughout our case studies, process blogging, and course materials. I also had an invitation to come on as an advisor on accessibility to UBC’s Emerging Media Lab, after I brought accessibility and my perspective on with a few of the students at their Autumn Showcase. This is an invitation I intend to follow up on this year, with the hope that between that and my own team’s process thinking through accessibility in extended reality, as we share extended reality case studies, work on our own extended reality development and process blog on it, and create our Extended Reality Development Series of Mini-Courses, that we also begin to build a standalone piece or pieces that address accessibility in extended reality. Not sure what that will be yet, but if you have suggestions for this, I’d love to hear about them in the comments.

Your Thoughts?

I’d love to hear your thoughts in comments below on the direction I am taking with this Extended Reality Open Education Guide. If you have suggestions for future XR Case Studies, on XR Development Tools and Platforms to try, the structure of or things I should be including in the XR Development Series of Mini-Courses, or ways to further address Accessibility in Extended Reality, please share them in the comments. Thank you!

Also, if anyone wishes to become involved in any of the above endeavours with myself and my team, please reach out and we can chat further.

Thank Yous

Thank You!

The work that I have begun here has been inspired by and is evolving, thanks to:

  • Dr David Vogt for encouraging my dreaming, entrepreneurial thinking, and edtech product design and development;
  • Dr David Porter in exploring open education and searching for sustainable funding pathways for;
  • Saeed Dyanatkar for seeing my potential in dreaming up different pathways for storytelling and education, always making time for me when I had questions, and making me feel welcome and valued whenever I wandered into the Emerging Media Lab;
  • Dr Kyle Stooshnov and Juliana Loh for creating a framework for my first steps into thinking about XR development and experimenting with 360 video;
  • Dr Jennifer Jenson and Dr Suzanne de Castell for fostering my game design thinking;
  • Dr Heidi Janz, Dr Michelle Stack and Dr Paul Hamilton for helping me to process my newer disabilities, frame disabilities and myself as disabled in my storytelling, and begin to understand how to advocate for disability and accessibility in education (and feel empowered to do so, understanding it as a strength, as oppose to something to hide);
  • Yvonne Dawydiak for demonstrating the alternative pathways to navigating UBC, approaching teaching creatively in a hands-on way, and an empathy driven approach and practice;
  • Lori Yearwood for her vision in beginning to create case studies here on StoryToGo to act as resources for our students, workshop participants, clients and partners, and the broader independent creative and education community – for encouraging me to begin to share my disabilities in our storytelling and work, and sharing her own wellness challenges as we began to build our health, wellness and accessibility workshops, resources, and storytelling – and for always being a willing and enthusiastic supporter and co-creator on the ideas that wander through my mind; and
  • Manami Calvo for seeing the potential for sharing her family’s WWII stories in AR, joining me in crafting experiments around, and in trusting me with her family’s stories.

Also an enormous thank you to the Interledger Foundation for helping fund my research and work on Sustainable Funding Solutions for Creatives and Educators, and to CNIE – RCIÉ and BCIT who have helped fund my research and coursework throughout my Masters.

Thank you all, for all your help, support, direction and inspiration!

Seeing the Invisible Augmented Reality Art Exhibit – an Extended Reality Case Study

by 4 Comments

In this second case study in our series exploring extended reality (XR), we examine the Seeing the Invisible Augmented Reality (AR) Art Exhibit, as exhibited by the Royal Botanical Gardens. This AR art exhibition featuring the art of 13 artists from around the world was initiated by the Jerusalem Botanical Gardens and displayed concurrently at 12 Botanical Gardens worldwide.

Project:

Seeing the Invisible Augmented Reality Art Exhibit

Initiated By:

Jerusalem Botanical Gardens in partnership with Outset Contemporary Art Fund, with the support of the Jerusalem Foundation

Curated By:

Hadas Maor and Tal Michael Haring

Host At:

Royal Botanical Gardens (RBG)

Type of Experience:

Augmented Reality (AR) Outdoor Art Exhibition

Device for Experiencing:

Smartphone or Tablet, preferably with headphones

Link to Project:

https://seeingtheinvisible.art/

Date Partook in the Experience:

September 18, 2022

Country of Origin:

Israel

Country Where I Experienced:

Canada

Reflections On My Experience

Interview Between Tucson Botanical Gardens Executive Director Michelle Conklin & Seeing The Invisible Curators Hadas Maor & Tal Michael Haring

The Project:

In 2021, as the world continued to face the pandemic and many art galleries were closed to the public, an idea was fostered out of the Jerusalem Botanical Gardens to host an outdoor art exhibit, but not in the usual sort of way. Rather an invisible art exhibit, at least to the naked eye, and one that was concurrently displayed in 12 gardens worldwide with the same pieces of original art. “How?”, you ask. With augment reality (AR), creating the first such multi-location, simultaneous art exhibition of its kind with a number of artists, many of whom were new to AR, creating their first pieces in augmented reality.

Some of the goals of this exhibition were to:

  • give visitors something new to enjoy in gardens at times of year when plants may not be blooming
  • complement the natural setting with the AR art, encouraging visitors to experience it in a new way
  • give people a way to enjoy art in a shared, yet outdoor setting
  • encourage visitors to the gardens to engage with the gardens and with the art
  • collaborate with other gardens worldwide
  • create an exhibition without disturbing the gardens themselves, and keeping the carbon footprint to a minimum

The art exhibition was also designed to address shared themes of “nature, environment, and sustainability, exploring the boundaries and connections between art, technology, and nature. Both bleak and hopeful, each artwork offering a unique perspective on unresolved issues, creating thought-provoking, experiential, and contemplative spaces for viewers to immerse in.”

The artwork and artists crafting this narrative included:

Despite the art exhibition taking place concurrently in a number of different gardens, the experience is different in each garden, as the works are augmenting the unique surroundings and context of each garden.

Why Augmented Reality?

Building and purchasing sculptures and other art installations is a pretty major undertaking for a botanical gardens, and a travelling exhibition can generally only visit one garden at a time. While not without a cost, by making this art exhibition simultaneously available in a number of botanical gardens, this makes the cost of such an art exhibition less prohibitive. The fact that this art exhibition is accessible through augmented reality via digital devices, makes it possible to ‘install the pieces’ temporarily in multiple gardens at the same time.

The initial launch of this AR art exhibition during the pandemic was well timed to offer an alternative outdoor way to enjoy art, outside the enclosed space of an art gallery, and further encouraged visitors to think about botanical gardens as arts spaces, as well as nature spaces.

The curators of Seeing the Invisible talk about the importance of this AR art exhibition being phygital, a blending of a digital experience with a physical one. Thought was put into where each artwork was placed in each garden to enhance both the artwork and the garden, as well as take visitor safety and the protection of the planted exhibitions into consideration. The AR artwork, through the Seeing the Invisible app, was geotagged to specific locations in the gardens they were exhibited in, and visitors could only experience the artwork in that location, and only in those gardens. Thanks to the phygital, the experience was unique in each garden.

The Augmented Reality Experience

The Seeing the Invisible artwork is experienced at the gardens hosting this AR art exhibition through visitors’ smartphones and tablets via a GPS triggered app.

Viewing AR art, Dawn Chorus, through a smartphone.

To see and experience these dynamic and engaging pieces of artwork, people visiting the gardens hosting this AR art exhibition need to:

  • Before visiting a Gardens hosting the AR art exhibition:
    • Download the Seeing the Invisible App to the smartphone or tablet with cellular capabilities that they will be using at the AR art exhibition.
    • Allow the app access to device’s camera and microphone.
    • Fully charge the device before visiting the gardens.
    • Take earbuds or headphones compatible with the device to the gardens with you.
  • At a Gardens hosting the AR art exhibition:
    • If you don’t have a smartphone or tablet, you can borrow one on site.
    • Follow the map in the app to the different pieces of AR art within the gardens.
    • Follow the Seeing the Invisible App’s instructions at each piece of AR art, scanning the ground where the piece of art has been virtually installed until the art virtually appears.

The Royal Botanical Gardens shares further tips on enjoying Seeing the Invisible Art Exhibition in their gardens, here.

Once the art virtually appears before you, visitors are encouraged to engage with it and fully immerse themselves in it, by listening through their headphones to the experience, and walking around the artwork and even into it. Visitors can also take pictures with the virtual art and read more about the art through the app.

Each piece of art, lends to different forms of engagement. For instance with:

  • Gilded Cage: You can walk into the labyrinth of cells.
  • Dawn Chorus: Birds swoop around you as you are drawn in by the music from the piano.
  • Biome Gateway: You can walk into the temple cave, and discover a portal to walk through and into a parallel landscape.
  • Forget Me Not: By moving toward the flower arrangement, you trigger a gun to propel a bullet through the flower arrangement causing it to explode outwards. Once the explosion has occurred, if you walk around the vase, you will hear three different scholars discussing the flower arrangement.
  • Pneuma: By walking into the sphere, you see the gardens around you through the distortion of being inside a bubble, while the sound of breathing acts to change your own breathing pattern while experiencing.

Morphecore Prototype AR even inspired a dancer to choreograph their own piece to dance along with. Other host Gardens similarly witnessed many of their visitors dancing along with the Morphecore Prototype.

My Reflections on this Augmented Reality Experience

One of my goals in exploring different extended reality experiences is to discover what excites me in these experiences and what I find challenging, in order to reflect on what could create even richer and more accessible extended reality experiences.

Here are my thoughts on the Seeing the Invisible AR Art Exhibition, from when I visited it in September of 2022 at the Royal Botanical Gardens.

Signs welcoming visitors to the Seeing the Invisible AR Art Exhibition at the Royal Botanical Gardens.

The Extended Reality Magic

This is a clever idea to get visitors exploring the gardens through a new lens, allowing the augmented reality art to spark their imagination in a new way, to perceive the gardens differently, and to provoke thought pertaining to nature, technology and art.

Some art, like Dawn Chorus, brought pure joy of the fantastically, magical whimsy that augmented reality brings with it, further adding beauty to the surrounding gardens.

AR birds flying around an augmented reality piano in a real world garden.
AR birds flying around an augmented reality piano, care of Dawn Chorus, in a real world garden.

Other art, like Pneuma, made you feel as though you’d stepped out of reality and were peering back in at it.

AR Art looking back at the gardens through the distortion of the augmented reality bubble created by Mel O’Callaghan's Pneuma.
Looking back at the gardens through the distortion of the augmented reality bubble created by Mel O’Callaghan’s Pneuma.

While other art, like Gilded Cage, got you questioning our problematically omnipotent and controlling relationship with nature from the perspective on an artist who was imprisoned.

AR art, Ai Weiwei's golden Gilded Cage, casting augmented reality shadows, by a park bench in a real world garden.
AR art of Ai Weiwei’s golden Gilded Cage, casting augmented reality shadows by a park bench in a real world garden.

Best of all, the gardens discovered that people of all ages wanted to engage with the art, even dancing along to Daito Manabe’s Morphecore Prototype AR.

A real world dancer at Jerusalem Botanical Gardens dancing alongside Daito Manabe's Morphecore Prototype AR.
A real world dancer at Jerusalem Botanical Gardens dancing alongside Daito Manabe’s Morphecore Prototype AR.

Logistically the GPS triggered app is a good idea to create a curated experience within each garden, and to host this simultaneously between different gardens.

Adding in the ability for visitors to be able to photograph the AR art through the app, added in the ability for visitors to have fun engaging with the art. This also added to crowdsourced storytelling around the exhibit, which I can only imagine was beneficial to marketing the exhibition.

Visitors at a number of gardens playing with Pneuma selfies.
Visitors at a number of gardens playing with Pneuma selfies.

I’d love to experience future such AR art exhibitions at the Royal Botanical Garden, as well as to experience the Seeing the Invisible Art Exhibition in different gardens to see how the experience differs between the gardens with the same art.

Current Challenges (as of 2022 Experience)

Excited to experience the Seeing the Invisible Art Exhibition at the Royal Botanical Gardens in Burlington, Ontario, my niece and I followed the instructions and downloaded the Seeing the Invisible App to our smartphones before heading to Hendrie Park to enjoy the exhibit together.

Seeing the Invisible at the Royal Botanical Garden's Hendrie Park.

The only problem, when we got there, the Seeing the Invisible App didn’t work on either of our smartphones, despite both having compatible technology. Not sure if this is the reason why, but one woman told us the issue was due to the strength of the wifi signal from your data provider.

No problem, we queued to borrow one of tablets that the gardens had available for visitors that needed it.

Borrowing a loaner tablet at the Royal Botanical Gardens.

While the borrowed tablet worked, this now meant that we were now sharing a device, our headphones didn’t work in the borrowed device, and the borrowed tablet had a screen protector over it, that made the AR more difficult to see, especially in sunny areas. Added to this, almost every piece of AR art was placed in a sunny area, and it was hot, so not only were we battling with the light to see the AR art through the screen, but needed shade for respite for the sun.

I assumed that this user experience oversight was down to this being the Royal Botanical Garden’s first AR Exhibition and them not being familiar enough with the technology to understand how to create a positive user experience, but it turns out it was actually the curators that mapped out where each piece would be placed. Maybe the oversight of the sun was made, as due to the pandemic, the curators did not visit Hendrie Park before deciding where to place the AR art, but either which way, this emphasizes the importance of taking the user experience into consideration when designing AR exhibitions. Shade is important, both for the visitors’ physical comfort and for the practicality of visitors actually being able to properly see and experience the AR art work through their screens. With the size of the Royal Botanical Gardens there is more than enough options to place these 13 pieces of artwork in a way that keeps visitors out of the full sun and allows them to better see and experience the AR art on their screens.

Girl holding a tablet, demonstrating the difficulty seeing the AR art in the sun.

While certainly the audio experience would have been much better had we been able to use our headphones, as people do experience exhibits like this on a shared device, I think it would be beneficial for the audio portion of the experience to be added at a higher volume to better allow for visitors sharing a device to be able to hear the experience.

I appreciated that the artist shared a write up on their art within the app, but to spend time reading this, it takes you away from viewing and experiencing the AR art. It would be great, if visitors have the additional option to play an audio recording of the artist sharing their thoughts with visitors on the piece of art.

Finally, with the loaned tablets, it would be beneficial if the individual gardens made it possible for visitors to email themselves the copies of any photos they shot with the AR art.

Takeaways From This Extended Reality Experience

When easily visible and heard, the Seeing the Invisible Art Exhibition creates a glimpse into a hidden world that encourages thought and exploration through a new lens and perspective. Thats exciting, and demonstrated what excites me about sharing stories with augmented reality.

However, if the app isn’t working on people’s devices or the AR is difficult to see on people’s screens, then people become frustrated, rather than excited by the experience. This is why user design and testing is so very important. This should be thought of both in terms of the technology and visitors’ interactions with the physical environment in which they are experiencing the AR art. In the case of the RBG, the gardens and not trampling plants seems to have been taken into considerations, but not visitors’ physical comfort from the sun or thought of how that sun would impact user’s experience. This art exhibition was almost in its entirety in direct sun with no to very little shade.

A girl in another full sun AR art exhibit spot at the Royal Botanical Gardens.

User testing should be inclusive of the loaner tablets for visitor use, made available onsite. This means, if adding protective screen covers to the tablets, making sure those protective screen covers do not hinder visitors’ ability to enjoy the AR art.

It would also be advantageous to allow people using the borrowed tablets to be able to email themselves any pictures they took with the AR art, or if that is not possible, to be upfront with visitors about that from the start, so as not to leave them disappointed at the end of the experience.

Summary of Takeaways:

  • The technology exists to create a GPS triggered augmented reality app that works on people’s devices without overloading a device’s data.
  • The Seeing the Invisible AR App takes up 2.1 GB of space on my smartphone.
  • The Seeing the Invisible AR App was developed by Khora ApS, a virtual reality and augmented reality production studio in Copenhagen.
  • When the AR app is working, the augmented reality art exhibition creates wonder for visitors, gets visitors exploring and interacting in a new way, and has visitors engaging with the AR art.
  • By making the AR art exhibition viewable through smartphones and tablets (with loaner tablets available for use), it is accessible to all ages and most abilities.
  • User testing is important on the site of the experience, thinking about user comfort at different times of year.
  • Avoid creating GPS triggered AR screen experiences in direct sun.
  • Make sure protective screen covers on loaner tablet do not limit the ability to see the augmented reality.
  • Think about user safety and comfort when placing GPS triggered AR experiences.
  • Encourage visitors to bring headphones, while ensuring sound is available at a volume that those without headphones or sharing a device can hear.
  • People enjoy interacting with AR art.
  • Make sure people can email themselves any pictures they took with the AR art on loaned tablets.

Future Building

In reflecting upon the Seeing the Invisible AR Art Exhibition, in addition to placing the augmented reality art in spaces that people can enjoy the art in the shade and to be better able to hear the accompanying sounds when experiencing through a shared device, I’d also love to further encourage the engagement that people enjoyed with the art. Some possible ideas for that:

  • I’ve recently seen how the Relive App makes it easy to create videos from a walk by recording the map and editing in the photos and videos you take at different stops. Should this be possible within this AR app, it could create fun keepsakes from the AR experience, that also become valuable crowdsourced storytelling for the exhibit. An example from the travels of a fellow member of Vancouver’s tech community:
  • It would be great to be able to play the artist’s description of their work as an audio reading, complete with some prompts to encourage people to engage with their art in different ways.
  • I loved how Ori Gersht’s Forget Me Not was responsive to visitor’s movements, it would be exciting to have more pieces that respond to the location of the holder of a device.
Ori Gersht's Forget Me Not AR art flower arrangement.
Exploding vase of flowers, as a part of Ori Gersht's Forget Me Not responsive AR art.

The educator in me has also begun to design, in my head, a scavenger hunt that encourages visitors to collect experiences and different perspectives with each piece of art by encouraging visitors to take on some sort of challenge or unravel some sort of mystery with each piece of art. Being able to create a mapped storytelling account of such a scavenger hunt, in a format like the Relive App creates, would be a great way to further share the discoveries from such a scavenger hunt.

The storyteller in me would also love to learn more about the making of the art, like with this talk on Ori Gersht’s and Timur Si-Qin’s AR art pieces.

What’s Your Take on this Augmented Reality Experience?

I’d love to hear your thoughts in comments below on what you think would make this a richer and more accessible AR experience, and if you have technical solutions for making this a more immersive and user friendly experience.

One visitor using an umbrella to improve his AR art experience at the Seeing the Invisible Art Exhibition at the RBG, with all the sun.
One visitor using a black umbrella to improve his AR art experience in the sun.

References

Conklin, M., [Tucson Botanical Gardens], Maor, H., & Haring, T. (2022, December 1). A Talk with the Curators of Seeing The Invisible [Video]. YouTube. https://www.youtube.com/watch?v=1BXqc0g8D_c

Horwitz, L. (2023, June 12). phygital. Customer Experience. https://www.techtarget.com/searchcustomerexperience/definition/phygital

Jerusalem Botanical Garden. (2021, November 14). אחת העבודות בתערוכת האמנות, תפתיע אתכם במיוחד. . .. Jerusalem Botanical Garden’s Facebook Page. https://www.facebook.com/Jerusalem.Botanical.Gardens/videos/372403381331281

Rendell, H., Gertler, C., Katri, M., Maor, H., & Haring, T. (2021). SEEING THE INVISIBLE. SEEING THE INVISIBLE. https://seeingtheinvisible.art/

Royal Botanic Garden Edinburgh, Maor, H., Gersht, O., & Si-Qin, T. (2022, May 24). Seeing the Invisible  In Conversation with the Artists [Video]. YouTube. https://www.youtube.com/watch?v=3_XD4nmV-58

Royal Botanical Gardens. (2023, March 8). Seeing the Invisible – Royal Botanical Gardens. https://www.rbg.ca/things-to-do/art-in-the-gardens/seeing-the-invisible

Shamir, R., [America-Israel Friendship League], Gertler, C., Rendell, H., Maor, H., Haring, T., Rominiecki, J., & Firestone, W. (2021, November 8). Seeing the Invisible: Augmented Reality art [Video]. YouTube. https://www.youtube.com/watch?v=TIuxE1oZKtY

Parco Archeologico di Tremona-Castello AR Experience – an Extended Reality Case Study

by 8 Comments

Welcome to our new series exploring extended reality case studies, in a quest to develop richer and more accessible Extended Reality (XR) experiences. In this first case study, we delve into Parco Archeologico di Tremona-Castello’s (Tremona Castello Archaeological Park) Augmented Reality (AR) Experience and speak with Nadia Lupi, one of the visionaries behind the project.

Organization: 

Parco Archeologico di Tremona-Castello (Tremona Castello Archaeological Park)

Project:

Parco Archeologico di Tremona-Castello AR Experience

Type of Experience:

Augmented Reality Walking Tour

Device for Experiencing:

Augmented Reality Glasses paired with a guided audio tour

Link to Project:

https://www.parco-archeologico.ch/en/visita/la-realta-virtuale.html

Interview / Partook in the Experience Date:

April 30, 2019

Interview With:

Nadia Lupi, Director of Mendrisiotto Turismo

Country:

Switzerland

Delving Into the Tremona Castello Archaeological Park AR Experience & an Interview with Nadia Lupi

The Project : Parco Archeologico di Tremona-Castello AR Experience

Parco Archeologico di Tremona-Castello

Sitting on top of the hillside over looking the village of Tremona in the Mendrisiotto District of the Canton of Ticino, Switzerland, local archaeologist Alfio Martinelli, discovered the archaeological remains of an ancient medieval village sharing a history of the area for 6,000 years from prehistory to the 14th century. As Alfio and other archaeologists and volunteers began to uncover this village, many questions arose as they began the puzzle together the stories from the past. Among the curiosities of Parco Archeologico di Tremona-Castello are why they chose to build there, with no apparent water source on site.

Who lived here? How did they spend their daily lives? Why did they suddenly abandon this village in the 14th century? These are all questions that Alfio, fellow archaeologists, and volunteers excavating Parco Archeologico di Tremona-Castello have been asking, as they piece together pictures and stories of daily life in this ancient village when it was thriving.

It is believed that this rural village was inhabited by artisans and farmers from dark ages until the 14th century, who travelled widely and traded their goods. The people there made widespread use of money and owned beautifully handcrafted ornamental items. The site was likely established for the village due to its strategic position, which allowed its inhabitants to control the communication routes from Como and Varese to Riva San Vitale, Monte Ceneri and the central Alps. Excavations have brought to light a labyrinthine and well-fortified settlement that offered its inhabitants refuge from Barbarian invasions and the skirmishes between Como and Milan.

Why Augmented Reality?

These are stories that Alfio and others excavating the site wanted to share, along with Mendrisiotto Turismo. Initially Alfio and Nadia Lupi, the Director of Mendrisiotto Turismo, envisioned turning Parco Archeologico di Tremona-Castello into a open air, living history museum.

They did not get approval for an open air, living history museum, however, so had to think of alternative ways of helping others to envision life in Parco Archeologico di Tremona-Castello in medieval times. Thanks to 3D Artist and Animator Elia Marcacci volunteering on the excavation of Parco Archeologico di Tremona-Castello, Alfio and Nadia got the idea to have Elia create 3D animations of life in Parco Archeologico di Tremona-Castello in medieval times, which gave them the idea to create an augmented reality experience that would allow visitors to Parco Archeologico di Tremona-Castello to envision the site in medieval times, while still seeing the present day excavation.

The Augmented Reality Experience

In creating this augmented reality experience, Alfio, Nadia and Elia worked with the University of Applied Sciences and Arts of Southern Switzerland (SUPSI) to help shape and technically guide this experience.

They wanted to create an experience that would allow visitors to envision and experience life in the ancient village, while still seeing the excavated foundations of the ancient walls on the site today.

Initially Nadia had imagined this being experienced in AR through a smartphone or a tablet, with ghostly walls of the village and ghostly figures of villagers wandering about around you, going about their daily work and life.

While they do not have that experience yet, they do have an 10 detailed 3D scenes into the past, beautifully illustrated by Elia, including places where you can visualise the ancient village all around you.

The Medieval Village that once stood at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.
The Medieval Village that once stood at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.

Bordering on Italy there was concern that if this experience were made accessible through smartphones and tablets, then visitors would run into issues with their devices accidentally switching to Italian telecommunication networks and resulting in additional charges. Thus the AR storytelling was created to be experienced through AR glasses instead.

To make this a richer experience, sharing audio storytelling about the ancient village and Parco Archeologico di Tremona-Castello, the AR glasses were paired with an audio guide that both shares stories on the 10-minute hike from the visitors centre to Parco Archeologico di Tremona-Castello and within Parco Archeologico di Tremona-Castello.

Due to the AR glasses and the audio guide, visitors can only partake in this experience when the visitor’s centre is open. You can get the address and hours of the visitor’s centre by clicking the button below, as well as details on renting the AR glasses and the audio guide.

Plan for an AR Walk through Parco Archeologico di Tremona-Castello

My Reflections on this Augmented Reality Experience

Part of my goal in exploring different extended reality experiences is to discover what excites me in these experiences and what I find challenging, in order to reflect on what could create even richer and more accessible extended reality experiences.

Here are my thoughts on Parco Archeologico di Tremona-Castello’s AR experience, from when I visited it in 2019.

The Extended Reality Magic

The concept video for Parco Archeologico di Tremona-Castello’s AR experience that Elia, Alfio and Nadia designed is beautiful and would make for an incredible experience for visitors, immersing them in two times / worlds simultaneously through ghostly echos from the past.

Elia’s 3D imagery is also well researched, well thought out, beautifully illustrated, and depicts incredible detail.

A household scene, as depicted by Elia Marcacci, in in the Medieval Village once at Parco Archeologico di Tremona-Castello.
A household scene, as depicted by Elia Marcacci, in in the Medieval Village once at Parco Archeologico di Tremona-Castello.

Based on memory, the audio storytelling was also well done, to transport visitors through time and help them to better imagine life in the ancient village at Parco Archeologico di Tremona-Castello.

Current Challenges (as of 2019 Experience)

As Nadia mentioned, the Parco Archeologico di Tremona-Castello’s AR experience is not yet as she envisioned it in their concept video. In 2019, rather than seeing an active ghostly village around you, you can visit a number of 3D graphic illustrations of life in the village that are generally static, aside from minor movement, like a wisp of smoke. While these 3D illustrations are well done, when you are expecting to walk into an active ghostly village, this is disappointing.

For me, however, the most problematic part of this AR experience is the AR glasses. They are uncomfortable, are difficult to see through, especially if you yourself wear glasses, and are difficult to see the 3D scenes through. As such, visitors don’t get to fully appreciate how well Elia’s illustrations have been crafted.

Erica wearing AR glasses at Tremona Castello Archaeological Park

While the audio storytelling is well designed, the audio storytelling device is problematic, as it has to be set in the visitor’s centre and once you start it, it cannot be paused or replayed without returning to the visitor’s centre to have it reset. This means you can’t take the time to enjoy and immerse yourself in the Parco Archeologico di Tremona-Castello’s AR experience at your own pace, but must stick to the pace of the device. This also means that on a hot, sunny day, you can’t take breaks from the tour under the shade of a tree, but must stay in the sun, until the tour reaches its completion.

The AR glasses and audio guide also limit when visitors can enjoy the experience to the hours of the Visitor’s Centre.

Takeaways From This Extended Reality Experience

Nadia’s initial description of what she envisioned for Parco Archeologico di Tremona-Castello’s AR experience is similar to what I’ve been envisioning for the natural and cultural history augmented reality experiences that I’ve been dreaming of. It is exactly the experience that I was hoping for, and I suspect that with advancements in motion capture suits and world building becoming more accessible through game design engines, like Unreal Engine and Unity, it is becoming easier to make visions like this a reality in the future, making me excited for a future visit to Parco Archeologico di Tremona-Castello to see how the AR experience has evolved. It would seem that Elia has already constructed and made it possible to explore the outdoor spaces of the ancient village that once stood at Parco Archeologico di Tremona-Castello through his 3D animations. Having said that, I think tackling this in pieces of starting with specific scenes, rather than a whole ghostly village all at once, makes this much more manageable logistically and financially, as well as allowing for the opportunity to test drive and learn from the initial scenes before designing more.

The AR glasses were very problematic for me, however, and create an accessibility challenge and I imagine a large expense in establishing this experience. I personally would design a version of this that visitors could enjoy through their smartphones and tablets. With advances in smartphone and tablet technology, the concerns for visitors’ devices switching to an Italian telecommunications network, resulting in unexpected cellular charges, could be resolved in the following ways:

  • Creating a GPS triggered app for the AR experience, that visitors download before leaving the visitor’s centre. Visitors can then turn on airplane mode on their devices, and start the app while using the visitor’s centre wifi. A number of travel audio guide apps work this way, including the popular GuideAlong audio tours.
  • Create a WebAR version of the AR experience, and install a mobile wifi router (like a starlink mobile) at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode.

Being able to enjoy this AR experience through a smartphone or tablet would make it accessible to more people, would make Elia’s 3D illustrations easier to view and enjoy, and would allow people to enjoy the experience at their own pace, pausing and replaying it, if they wish. In this way, the AR experience could even be made accessible outside of the visitor’s centre’s hours, if so desired.

It should be noted that one advantage to AR glasses over WebAR is that the augmented realities resolution is usually better with AR glasses, but if the augmented reality is difficult to see through the AR glasses, then it doesn’t really matter how good the resolution is.

Food storage in the Medieval Village once at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.
Food storage in the Medieval Village once at Parco Archeologico di Tremona-Castello, as depicted by Elia Marcacci.

Summary of Takeaways:

  • creating a ghostly ancient village that can be viewed with augmented reality with ghostly figures actively taking part in village life is where the magic in this experience is for me
  • creating a series of ghostly scenes from the village is a more manageable starting point, and allows for the opportunity to test drive and learn from the initial scenes before designing more
  • AR glasses are uncomfortable for some and difficult for some people to see the augmented reality scenes in
  • AR and audio storytelling is more accessible when experienced through a smartphone or a tablet
  • concerns of smartphones or tablets switching to an Italian telecommunications network, could be resolved by having the AR experience accessible through a GPS triggered app or available through WebAR if a mobile wifi router was available for use at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode
  • being able to enjoy this AR experience through a smartphones or tablet would allow people to immerse themselves in the experience at their own pace, pausing and replaying it, if they wish

Future Building

In reflecting upon Parco Archeologico di Tremona-Castello’s AR experience, in addition to the desire to have this experience accessible through a smartphone or table, allowing me to pause and replay the audio storytelling, and to see the ancient village in ghostly form with active ghostly villagers, I’d also love to be able to interact with those scenes in some way. Some possible ideas for that:

  • the ability to take pictures with the ghostly village and villagers, including selfies
  • being able to listen in on the conversations of ghostly villagers
  • being able to learn an activity from the ghostly villagers, perhaps as they teach a ghostly child or apprentice
  • being able to partake in a festive event, by dancing or signing along with the ghostly villagers

The educator in me is also envisioning creating the village in VR for use in school classrooms, or possibly VR (or AR) teaser experiences that would get school kids and other visitors prepared for a visit with a bit of a background of the time period that they will be transported to and some questions or things to look out for and think about on their visit to Parco Archeologico di Tremona-Castello.

Exploring Tremona-Castello Archaeological Park with AR glasses.

What’s Your Take on this Augmented Reality Experience?

I’d love to hear your thoughts in comments below on what you think would make this a richer and more accessible AR experience, and if you have technical solutions for making this a more immersive and user friendly experience.

References

Hargreave, E., [StoryToGo]. (2023b, December 15). Tremona Castello Archaeological Park AR Experience [Video]. YouTube. https://www.youtube.com/watch?v=6f8NZaeC_do

La storia. (n.d.). https://aram-ti.ch/il-villaggio/la-storia/

Marcacci Elia. (2014, January 6). Ein Tag in Tremona-Castello. Medienarchiv. https://medienarchiv.zhdk.ch/entries/63853132-1225-498b-ba50-c0b38d3a9eb9

Martinelli, A. (2008). Tremona Castello. Dal V millennio a.C. al XIII secolo d.C. All’Insegna del Giglio.

MendrisiottoTurismo. (2016a, May 25). Il villaggio di Tremona-Castello in 3D [Video]. YouTube. https://www.youtube.com/watch?v=4gzp_A9ADrQ

Mendrisiotto Turismo. (2016b, August 10). Parco archeologico di Tremona-Castello, oggi [Video]. YouTube. https://www.youtube.com/watch?v=ri7HMuhSrVQ

MendrisiottoTurismo. (2016c, August 29). Anno Domini – Rievocazione storica al Parco archeologio di Tremona-Castello [Video]. YouTube. https://www.youtube.com/watch?v=hkCdTiknS-4

MendrisiottoTurismo. (2019a, May 15). Parco Archeologico di Tremona Castello [Video]. YouTube. https://www.youtube.com/watch?v=qHGKHO4bZBI

MendrisiottoTurismo. (2019b, July 12). Hunting for mysteries in time [Video]. YouTube. https://www.youtube.com/watch?v=Ty9LxvWVsOw

Mendrisiotto Turismo. (2023, May 19). La Regione da scoprire – Il Parco archeologico di Tremona 2023 [Video]. YouTube. https://www.youtube.com/watch?v=6BbQJVkru30

Parco Archeologico di Tremona-Castello (n.d.). Ticino Tourism. https://www.parco-archeologico.ch/en/

Società editrice del Corriere del Ticino SA. (n.d.). Tremona – TicinoTopTen. Ticino Top Ten. https://www.ticinotopten.ch/en/experiences/tremona

Exploring XR Development with my Final Master of Educational Technology Directed Studies

by 4 Comments

Join me in exploring XR Development for my final Master of Educational Technology Directed Studies, and finding ways of making XR Development more manageable and accessible for other educators, students and creatives navigating XR and beginning to develop their own projects.

One of my main goals in embarking upon my Masters of Educational Technology was to explore newer-to-me forms of storytelling that could help to bring education to life. Over the past 7 years as I’ve worked my way through my Masters, being introduced to UBC’s Emerging Media Lab and the work they are doing there, having been approached by a board member from the Juno Beach Centre to explore newer ways of sharing stories from World War II that would connect with Canadian school children, travelling to Normandy to visit the museum and see how they are sharing stories there and identifying the gaps in their storytelling, and being shortlisted and interviewed for an interactive and immersive storytelling fellowship in Norway and exploring the stories I might wish to create for that fellowship, the idea of delving deeper into education and storytelling through XR – Extended Reality, teased at my synapses. I took an early course in VR in Education, attended XR symposiums and showcases, began researching and exploring different applications of XR in education and storytelling, collaborated on an open course in Immersive Experiences in Natural and Cultural History Education, developed an idea for an AR storytelling app to share natural and cultural history experiences, and am working my way through other people’s courses in XR, including the University of Michigan’s Extended Reality series of courses. While I am only now beginning to write about all of what I have learned here, the series of posts that follow come from research and a rabbit hole that I began to travel through 6-years-ago, and to date have included four talks at national and international conferences in education, technology, and storytelling, sharing my findings and explorations, and one talk to a Journalism Masters Class at the American University of Cairo. I have shared that last talk below as it shares many of the goals that I set out to accomplish with this directed studies.

Defining Immersive and Interactive Media

Before we get into where my interests and observations are taking me with my final directed studies for my Masters, I thought we’d explore the terminology around immersive media, experiential media, and extended reality, as there is ambiguity around the terminology, so by addressing this at the onset, I hope to at least make it clear as to how I am using the terminology.

An Immersive art exhibit entitled, Interactive Diorama - Rembrandt 1632.
Interactive Diorama—Rembrandt, 1632, The Anatomy Lesson of Dr. Nicolaes Tulp / Aalto University, School of Arts, Design and Architecture, Helsinki. Photographed by Tom Mesic.

Immersive and Interactive Media / Experiential Media

Immersive and Interactive Media are basically the current term for what was termed transmedia, referring to media that is immersive and interactive in nature, allowing those that are engaged with it to experience it in a way that allows them to step into the experience and / or become a part of the media and / or interact with the media, potentially having an impact on how the experience is shaped.

Examples of forms of immersive and interactive media include virtual reality, augmented reality, escape rooms, 360 video, choose your own adventure stories, alternate reality games … and the list goes on. Another name for this form of media is experiential media

Extended Reality (XR)

Project Lead Olivia How, demonstrating Shakespeare xR's touch table of Shakespeare’s First Folio to interested onlookers.

Extended Reality is a form of immersive / experiential media that takes users into a new reality or places virtual objects into a user’s real world, enhancing everyday life with technology. Basically making impossible sensorial experiences, possible. (Hargreave, 2021a)

Included within extended reality are virtual reality (VR), augmented reality (AR), and mixed reality (MR).

Virtual Reality (VR)

Virtual Reality is a computer-generated environment with scenes and objects that appear to be real, making the user feel they are immersed in their surroundings. (Iberdrola, n.d.) While this often is experienced through the use of a headset, a user could also walk into a virtual reality environment created by projectors and screens. This simulated environment and experience could be similar to or completely different from that of the real world in which a user is physically situated.

Examples of virtual reality include 360 video, when experienced through a headset, meeting in Mozilla Hubs, and the Immersive Van Gogh Exhibit.

A student experimenting with the Occulus Quest in the Emerging Media Lab at UBC.
A student experimenting in VR at the Emerging Media Lab at UBC.

Augmented Reality (AR)

Augmented Reality (AR) is an interactive experience where the real, physical world is digitally augmented. The augmentation is usually visual in current applications, but it can also be through other senses, such as auditory or haptic. (Wong, 2021)

Examples of augmented reality include Pokemon Go, the ReBlink exhibition at the AGO, and selfie masking filters in SnapChat.

Exploring Tremona-Castello Archaeological Park with AR glasses.
Exploring Tremona-Castello Archaeological Park with AR glasses.

Mixed Reality (MR)

There is a lot of debate around what constitutes Mixed Reality. I have always thought of it and defined it as allowing for real and virtual elements to interact with one another, and for users to interact with virtual elements in a similar way they would in the real world or at least in a similar way that they might interact with a touch screen. (Hargreave, 2021b)

Students observing the Holobrain at UBC.
Observing the Holobrain at UBC, care of Kerry Blackadar.

Others have defined mix reality (Speicher et al, 2019) as:

  • a synonym for augmented reality
  • a combination of AR and VR
  • a stronger version of AR
  • a type of collaboration
  • the idea of aligning physical virtual environments. 
  • the spectrum from the real environment without any augmentation to the virtual environment, which is completely synthesized computer-generated virtual content. 

This is further confused by some researchers only working within the theoretical and not experimenting with the tools in real time and some corporations attempting to use the term ‘mixed reality’ to market the next level advances in extended reality.

Due to all the debate around defining mixed reality, and based on one research paper, the unlikelihood of a consensus between the various experts in research and industry, I will likely avoid the term mixed reality and instead describe the characteristics of various extended reality applications, and possibly where they lie on the reality–virtuality continuum developed by Paul Milgram and colleagues in 1994.

Reality-Virtuality Continuum

The Reality-Virtuality Continuum as proposed by Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino in their 1994 paper, encompasses all possible variations and compositions of real and virtual objects.  They hypothesized that everything between real and the virtual is mixed reality, and that mixed reality makes up both augmented reality and augment virtuality. (Milgram et al, 1994) Basically suggesting that mixed reality is the blending of the physical and the virtual world with varying degrees of augmentation. 

The problem with the RV Continuum in its current form is that it only takes into account visual, not sound, olfactory or haptics senses. (Speicher et al, 2019) It ultimately needs to be redeviced with how technologies are evolving.

Augmented Virtuality

Augmented Virtuality refers to perceiving mostly virtual content, while still seeing some real world content. (Nebeling, 2022)

My Interests and Explorations in Extended Reality

Virtual Reality

I have long been fascinated with extended reality, ever since I was first introduced to the concept of virtual reality in an episode of Murder She Wrote in 1993. It excited me to imagine how extended reality could be used to create immersive story experience and bring worlds to life that are created in one’s imagination. The problem for me though was that whenever I tried it, which I did over the years whenever an opportunity present itself, was that it was too much sensory overload for me, causing mr to feel motion sick (which I now know is cybersickness) and sparked migraines. (Weech et al, 2019)

Jessica Fletcher in a VR headset as she solves a 1993 Murder She Wrote mystery.
Jessica Fletcher in a VR headset as she solves a 1993 Murder She Wrote mystery.

Persistence pays off though, as in 2018 I finally had a VR experience that did not spark cybersickness or a migraine. The experience in question was a virtual plank walk designed to help people overcome their fear of heights. (Basbasse et al, 2023) In exploring the features that made the difference in this experience for me and talking to one of the designers of the experience, I believe the quality of the filming / graphics made the main difference in this experience, as well as perhaps the experience being designed to limit your time in VR. This also got me inquiring from the designer if VR was being used in other ways in the medical world, and if perhaps it was being used with pain management. My interest? By this point I’d been living with chronic pain following two car accidents for 5 years, and was curious if perhaps there were ways this technology could be used to help people better manage their pain. This sparked me to begin explore different medical avenues and uses to VR, as I began to contemplate what might be useful to me and to the community of chronic pain and fatigue sufferers that I now belonged to, many of whom are predominantly home bound.

VR Plank Walk
Epionia Therapeutic VR
Therapeutic VR for Pain Reduction

Augmented Reality

I have always loved and been drawn to educational experiences that incorporate storytelling and encourage the imagination. This was developed in me as a parks naturalist and is part of how I approached science education, and encourage kids to think, imagine, experience and explore.

In approaching work and storytelling, post the car accidents that very much changed my life, how I work, what I can handle and in some ways who I am, I have been increasingly drawn back towards natural and cultural history in my storytelling, as nature soothes my symptoms. When I travel I also love learning and love to envelop myself in the story of a space, but thanks to my post accidents realities I cannot always handle busy museums with noise, lights, and other sensory triggers. This got me imagining, first with my fellowship interview in Norway and later as I explored the Juno Beach Centre in Normandy, if I could create mobile storytelling experiences outside of the museums in places of natural and cultural history, where people could learn and enjoy the stories of the place as they explored outdoors. In both cases, as they were looking to me to imagine newer ways of bringing such stories to life, I began to imagine this with augmented reality as experienced through our mobile devices. This made me smile, as not only did this idea hold the potential for learning and storytelling, but also some of the fun, whimsy and magic that augmented reality brings with it. This is an idea that I further began to sketch out on my travels and in my ETEC 522 Course – Ventures in Educational Technology, and that I am working towards making a reality.

My Goals with this Directed Studies into Extended Reality

I have a tendency to dream big and my dyslexic brain sees connections and expanding and overlapping storyworlds. Usually this means that I do not have the money and often all the knowledge for orchestrating all that I imagine at the get go. My dyslexia has taught me to problem solve though, and life and an imagination and brain that don’t let go of ideas have taught me patience and to play the long game, approaching my ideas in digestible pieces. Thats what I have been doing with this directed studies. I essentially started the work on it after the first year into my Masters, slowly taking advantage of research experiences as they became available, participating in XR symposiums and showcases when they present themselves, taking formal and informal courses in XR, accepting opportunities to speak on what I am working towards with XR and learning along the way, and beginning to flush out my ideas and pitch them to funding bodies.

For this directed studies, my goals are to:

  • Write up the VR case studies that are helping me to identify the strengths and challenges in VR for patient communities.
  • Write up the AR case studies that are helping me to identify the strengths and challenges in AR for natural and history education.
  • Begin to develop and further experiment with VR and AR development.
  • Sharing what I have learned to help make it more manageable and accessible for other educators, students and creatives navigating XR and begin to develop their own projects.

A Beginning

While in some ways I see this directed studies as an end to my researching and a beginning to my developing, there will always be more research to be done in seeing how others are developing and applying the XR, especially as the technology is still developing and despite its decades of history, still very much in its early days for development, application and adoption.

Development-wise, waiting on gear and difficulties with my health and post concussion syndrome, have very much slowed down my intended goals with development and where I’d like to be at this stage, in terms of developing my own XR applications in VR for patients and AR for natural and cultural history. However, as my post accidents self has learned, pushing through when my health issues becomes problematic, only making things worse and setting me back further. Meaning that I have had to be patient with myself, have had to get my health back to a stable place (during a year that posed a lot of challenges with that), and have had to address the backlog of pressing deadlines from my health going sideways before I’ve been able to engage in development. I’ve put in the work to do that, and am now in a good and healthy space to embark on development. This is much later than I would have liked, but that is not something I can change with my newer disability realities post the car accidents. As such, my initial development goals for this directed studies will be smaller for now, and I will continue working on the bigger goals, post the directed studies and will continue to write those up as I do.

Erica wearing AR glasses at Tremona Castello Archaeological Park

This also means that for the Guide to XR Development that I will be developing to make XR Development more manageable and accessible for other educators, students and creatives, this will similarly take shape in stages and over time, some coming sooner with my directed studies and some continuing over time, as I further delve into XR development and experiments of my own.

Thank you for joining me on this journey into XR Development, and being patient with me as I take the time to do this in a way that is healthy for me with my health and disability realities. Hopefully my journey as the tortoise on this road will inspire others to explore and embark on such a journey of their own, no matter at what speed and working with whatever challenges your life and situation might pose along the way.

References

El Basbasse, Y., Packheiser, J., Peterburs, J., Maymon, C., Güntürkün, O., Grimshaw, G., & Ocklenburg, S. (2023). Walk the plank! Using mobile electroencephalography to investigate emotional lateralization of immersive fear in virtual reality. Royal Society open science, 10(5), 221239. https://doi.org/10.1098/rsos.221239

Hargreave, E. (2021a, November 15). An introduction to immersive experiences in natural & cultural history education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/intro/

Hargreave, E. (2021b, November 15). Mixed Reality (MR) in Natural & Cultural History Education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/mr-in-natural-cultural-history-education/

Hargreave, E., [Stories, EdTech & Digital Media with Ahimsa Media]. (2020, November 29). Story Steppers – an AR storytelling app to the natural & cultural history of a place [Video]. YouTube. https://www.youtube.com/watch?v=2xeWjBWYcAI

Hargreave, E., [StoryToGo]. (2023, December 10). Exploring Immersive Media to Develop Creative & Engaging Stories with Master of Journalism Students [Video]. YouTube. https://www.youtube.com/watch?v=Ff1cMMiH5cc

Iberdrola. (n.d.). Virtual Reality: another world within sight. Retrieved December 7, 2023, from https://www.iberdrola.com/innovation/virtual-reality

P. Milgram, H. Takemura, A. Utsumi, F. Kishino: Augmented Reality: A class of displays on the reality-virtuality continuum. In Proceedings of Telemanipulator and Telepresence Technologies, 1994.

Nebeling, M. (2022, June 2). Intro to AR/VR/MR/XR: Technologies, applications & issues. Coursera. https://www.coursera.org/learn/intro-augmented-virtual-mixed-extended-reality-technologies-applications-issues

M. Speicher, B.D. Hall, M. Nebeling: What is Mixed Reality? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 4-9, 2019.

Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Frontiers in psychology, 10, 158. https://doi.org/10.3389/fpsyg.2019.00158

Wong, Y. (2021, November 15). Augmented Reality (AR) in Natural & Cultural History Education – StoryToGo Classroom. StoryToGo Classroom. https://storytogo.ca/classroom/course/immersive-experiences-in-natural-and-cultural-history/lessons/ar/

StoryToGo