Welcome to our new series exploring extended reality case studies, in a quest to develop richer and more accessible Extended Reality (XR) experiences. In this first case study, we delve into Parco Archeologico di Tremona-Castello’s (Tremona Castello Archaeological Park) Augmented Reality (AR) Experience and speak with Nadia Lupi, one of the visionaries behind the project.
Organization:
Parco Archeologico di Tremona-Castello (Tremona Castello Archaeological Park)
Project:
Parco Archeologico di Tremona-Castello AR Experience
Type of Experience:
Augmented Reality Walking Tour
Device for Experiencing:
Augmented Reality Glasses paired with a guided audio tour
Interview / Partook in the Experience Date:
April 30, 2019
Interview With:
Nadia Lupi, Director of Mendrisiotto Turismo
Country:
Switzerland
Delving Into the Tremona Castello Archaeological Park AR Experience & an Interview with Nadia Lupi
The Project : Parco Archeologico di Tremona-Castello AR Experience
Parco Archeologico di Tremona-Castello
Sitting on top of the hillside over looking the village of Tremona in the Mendrisiotto District of the Canton of Ticino, Switzerland, local archaeologist Alfio Martinelli, discovered the archaeological remains of an ancient medieval village sharing a history of the area for 6,000 years from prehistory to the 14th century. As Alfio and other archaeologists and volunteers began to uncover this village, many questions arose as they began the puzzle together the stories from the past. Among the curiosities of Parco Archeologico di Tremona-Castello are why they chose to build there, with no apparent water source on site.
Who lived here? How did they spend their daily lives? Why did they suddenly abandon this village in the 14th century? These are all questions that Alfio, fellow archaeologists, and volunteers excavating Parco Archeologico di Tremona-Castello have been asking, as they piece together pictures and stories of daily life in this ancient village when it was thriving.
It is believed that this rural village was inhabited by artisans and farmers from dark ages until the 14th century, who travelled widely and traded their goods. The people there made widespread use of money and owned beautifully handcrafted ornamental items. The site was likely established for the village due to its strategic position, which allowed its inhabitants to control the communication routes from Como and Varese to Riva San Vitale, Monte Ceneri and the central Alps. Excavations have brought to light a labyrinthine and well-fortified settlement that offered its inhabitants refuge from Barbarian invasions and the skirmishes between Como and Milan.
Why Augmented Reality?
These are stories that Alfio and others excavating the site wanted to share, along with Mendrisiotto Turismo. Initially Alfio and Nadia Lupi, the Director of Mendrisiotto Turismo, envisioned turning Parco Archeologico di Tremona-Castello into a open air, living history museum.
They did not get approval for an open air, living history museum, however, so had to think of alternative ways of helping others to envision life in Parco Archeologico di Tremona-Castello in medieval times. Thanks to 3D Artist and Animator Elia Marcacci volunteering on the excavation of Parco Archeologico di Tremona-Castello, Alfio and Nadia got the idea to have Elia create 3D animations of life in Parco Archeologico di Tremona-Castello in medieval times, which gave them the idea to create an augmented reality experience that would allow visitors to Parco Archeologico di Tremona-Castello to envision the site in medieval times, while still seeing the present day excavation.
The Augmented Reality Experience
In creating this augmented reality experience, Alfio, Nadia and Elia worked with the University of Applied Sciences and Arts of Southern Switzerland (SUPSI) to help shape and technically guide this experience.
They wanted to create an experience that would allow visitors to envision and experience life in the ancient village, while still seeing the excavated foundations of the ancient walls on the site today.
Initially Nadia had imagined this being experienced in AR through a smartphone or a tablet, with ghostly walls of the village and ghostly figures of villagers wandering about around you, going about their daily work and life.
While they do not have that experience yet, they do have an 10 detailed 3D scenes into the past, beautifully illustrated by Elia, including places where you can visualise the ancient village all around you.
Bordering on Italy there was concern that if this experience were made accessible through smartphones and tablets, then visitors would run into issues with their devices accidentally switching to Italian telecommunication networks and resulting in additional charges. Thus the AR storytelling was created to be experienced through AR glasses instead.
To make this a richer experience, sharing audio storytelling about the ancient village and Parco Archeologico di Tremona-Castello, the AR glasses were paired with an audio guide that both shares stories on the 10-minute hike from the visitors centre to Parco Archeologico di Tremona-Castello and within Parco Archeologico di Tremona-Castello.
Due to the AR glasses and the audio guide, visitors can only partake in this experience when the visitor’s centre is open. You can get the address and hours of the visitor’s centre by clicking the button below, as well as details on renting the AR glasses and the audio guide.
My Reflections on this Augmented Reality Experience
Part of my goal in exploring different extended reality experiences is to discover what excites me in these experiences and what I find challenging, in order to reflect on what could create even richer and more accessible extended reality experiences.
Here are my thoughts on Parco Archeologico di Tremona-Castello’s AR experience, from when I visited it in 2019.
The Extended Reality Magic
The concept video for Parco Archeologico di Tremona-Castello’s AR experience that Elia, Alfio and Nadia designed is beautiful and would make for an incredible experience for visitors, immersing them in two times / worlds simultaneously through ghostly echos from the past.
Elia’s 3D imagery is also well researched, well thought out, beautifully illustrated, and depicts incredible detail.
Based on memory, the audio storytelling was also well done, to transport visitors through time and help them to better imagine life in the ancient village at Parco Archeologico di Tremona-Castello.
Current Challenges (as of 2019 Experience)
As Nadia mentioned, the Parco Archeologico di Tremona-Castello’s AR experience is not yet as she envisioned it in their concept video. In 2019, rather than seeing an active ghostly village around you, you can visit a number of 3D graphic illustrations of life in the village that are generally static, aside from minor movement, like a wisp of smoke. While these 3D illustrations are well done, when you are expecting to walk into an active ghostly village, this is disappointing.
For me, however, the most problematic part of this AR experience is the AR glasses. They are uncomfortable, are difficult to see through, especially if you yourself wear glasses, and are difficult to see the 3D scenes through. As such, visitors don’t get to fully appreciate how well Elia’s illustrations have been crafted.
While the audio storytelling is well designed, the audio storytelling device is problematic, as it has to be set in the visitor’s centre and once you start it, it cannot be paused or replayed without returning to the visitor’s centre to have it reset. This means you can’t take the time to enjoy and immerse yourself in the Parco Archeologico di Tremona-Castello’s AR experience at your own pace, but must stick to the pace of the device. This also means that on a hot, sunny day, you can’t take breaks from the tour under the shade of a tree, but must stay in the sun, until the tour reaches its completion.
The AR glasses and audio guide also limit when visitors can enjoy the experience to the hours of the Visitor’s Centre.
Takeaways From This Extended Reality Experience
Nadia’s initial description of what she envisioned for Parco Archeologico di Tremona-Castello’s AR experience is similar to what I’ve been envisioning for the natural and cultural history augmented reality experiences that I’ve been dreaming of. It is exactly the experience that I was hoping for, and I suspect that with advancements in motion capture suits and world building becoming more accessible through game design engines, like Unreal Engine and Unity, it is becoming easier to make visions like this a reality in the future, making me excited for a future visit to Parco Archeologico di Tremona-Castello to see how the AR experience has evolved. It would seem that Elia has already constructed and made it possible to explore the outdoor spaces of the ancient village that once stood at Parco Archeologico di Tremona-Castello through his 3D animations. Having said that, I think tackling this in pieces of starting with specific scenes, rather than a whole ghostly village all at once, makes this much more manageable logistically and financially, as well as allowing for the opportunity to test drive and learn from the initial scenes before designing more.
The AR glasses were very problematic for me, however, and create an accessibility challenge and I imagine a large expense in establishing this experience. I personally would design a version of this that visitors could enjoy through their smartphones and tablets. With advances in smartphone and tablet technology, the concerns for visitors’ devices switching to an Italian telecommunications network, resulting in unexpected cellular charges, could be resolved in the following ways:
- Creating a GPS triggered app for the AR experience, that visitors download before leaving the visitor’s centre. Visitors can then turn on airplane mode on their devices, and start the app while using the visitor’s centre wifi. A number of travel audio guide apps work this way, including the popular GuideAlong audio tours.
- Create a WebAR version of the AR experience, and install a mobile wifi router (like a starlink mobile) at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode.
Being able to enjoy this AR experience through a smartphone or tablet would make it accessible to more people, would make Elia’s 3D illustrations easier to view and enjoy, and would allow people to enjoy the experience at their own pace, pausing and replaying it, if they wish. In this way, the AR experience could even be made accessible outside of the visitor’s centre’s hours, if so desired.
It should be noted that one advantage to AR glasses over WebAR is that the augmented realities resolution is usually better with AR glasses, but if the augmented reality is difficult to see through the AR glasses, then it doesn’t really matter how good the resolution is.
Summary of Takeaways:
- creating a ghostly ancient village that can be viewed with augmented reality with ghostly figures actively taking part in village life is where the magic in this experience is for me
- creating a series of ghostly scenes from the village is a more manageable starting point, and allows for the opportunity to test drive and learn from the initial scenes before designing more
- AR glasses are uncomfortable for some and difficult for some people to see the augmented reality scenes in
- AR and audio storytelling is more accessible when experienced through a smartphone or a tablet
- concerns of smartphones or tablets switching to an Italian telecommunications network, could be resolved by having the AR experience accessible through a GPS triggered app or available through WebAR if a mobile wifi router was available for use at Parco Archeologico di Tremona-Castello for visitors to use while their devices are in airplane mode
- being able to enjoy this AR experience through a smartphones or tablet would allow people to immerse themselves in the experience at their own pace, pausing and replaying it, if they wish
Future Building
In reflecting upon Parco Archeologico di Tremona-Castello’s AR experience, in addition to the desire to have this experience accessible through a smartphone or table, allowing me to pause and replay the audio storytelling, and to see the ancient village in ghostly form with active ghostly villagers, I’d also love to be able to interact with those scenes in some way. Some possible ideas for that:
- the ability to take pictures with the ghostly village and villagers, including selfies
- being able to listen in on the conversations of ghostly villagers
- being able to learn an activity from the ghostly villagers, perhaps as they teach a ghostly child or apprentice
- being able to partake in a festive event, by dancing or signing along with the ghostly villagers
The educator in me is also envisioning creating the village in VR for use in school classrooms, or possibly VR (or AR) teaser experiences that would get school kids and other visitors prepared for a visit with a bit of a background of the time period that they will be transported to and some questions or things to look out for and think about on their visit to Parco Archeologico di Tremona-Castello.
What’s Your Take on this Augmented Reality Experience?
I’d love to hear your thoughts in comments below on what you think would make this a richer and more accessible AR experience, and if you have technical solutions for making this a more immersive and user friendly experience.
References
Hargreave, E., [StoryToGo]. (2023b, December 15). Tremona Castello Archaeological Park AR Experience [Video]. YouTube. https://www.youtube.com/watch?v=6f8NZaeC_do
La storia. (n.d.). https://aram-ti.ch/il-villaggio/la-storia/
Marcacci Elia. (2014, January 6). Ein Tag in Tremona-Castello. Medienarchiv. https://medienarchiv.zhdk.ch/entries/63853132-1225-498b-ba50-c0b38d3a9eb9
Martinelli, A. (2008). Tremona Castello. Dal V millennio a.C. al XIII secolo d.C. All’Insegna del Giglio.
MendrisiottoTurismo. (2016a, May 25). Il villaggio di Tremona-Castello in 3D [Video]. YouTube. https://www.youtube.com/watch?v=4gzp_A9ADrQ
Mendrisiotto Turismo. (2016b, August 10). Parco archeologico di Tremona-Castello, oggi [Video]. YouTube. https://www.youtube.com/watch?v=ri7HMuhSrVQ
MendrisiottoTurismo. (2016c, August 29). Anno Domini – Rievocazione storica al Parco archeologio di Tremona-Castello [Video]. YouTube. https://www.youtube.com/watch?v=hkCdTiknS-4
MendrisiottoTurismo. (2019a, May 15). Parco Archeologico di Tremona Castello [Video]. YouTube. https://www.youtube.com/watch?v=qHGKHO4bZBI
MendrisiottoTurismo. (2019b, July 12). Hunting for mysteries in time [Video]. YouTube. https://www.youtube.com/watch?v=Ty9LxvWVsOw
Mendrisiotto Turismo. (2023, May 19). La Regione da scoprire – Il Parco archeologico di Tremona 2023 [Video]. YouTube. https://www.youtube.com/watch?v=6BbQJVkru30
Parco Archeologico di Tremona-Castello (n.d.). Ticino Tourism. https://www.parco-archeologico.ch/en/
Società editrice del Corriere del Ticino SA. (n.d.). Tremona – TicinoTopTen. Ticino Top Ten. https://www.ticinotopten.ch/en/experiences/tremona
David Vogt says
Thanks for this thoughtful, balanced analysis, Erica. You’re right: the key tension here is that AR glasses are simply not yet up to the task of seamlessly overlaying virtual experience onto real world experience, especially in daylight conditions. Given the multiple challenges of latency, matching light levels, accurate mapping of position and perspective, etc., I can’t imagine glasses will achieve engaging suspension-of-disbelief anytime soon, although the invaluable “magic” they could deliver is a huge driver for developers trying to make them work. Meanwhile, smartphones and tablets are an acceptable compromise.
I’m reminded of a project I undertook in Vancouver more than 15 years ago – in the area of ‘forensic filmography’ – with the concept that with the number of films shot in the city one could generate a tourist attraction by taking them to the real-world locations of specific scenes, allowing them to view those scenes there on their smartphones, and then encouraging them to use their smartphones to capture them re-enacting the scenes as they were shot, with the same shot angles, etc. You could consider it ‘movie karaoke’. This project never achieved full funding, but I consider it even more viable now.
My point is that, as far as I can tell, Tremona-Castello has only consider the pre-experience and live-experience AR possibilities, not the post-experience values. For example, with today’s technology you could encourage visitors to use their smartphones to video themselves walking through the archaeological park, perhaps after having been given an imagined medieval task to do, and then these videos could be augmented by a post-production engine to fill in the walls, the set-dressings, the costumes, etc., so that it is the visitors themselves who are in the scenes, and they can take these home. I doubt the technology is fast enough yet to do this “live”, as in the person holding the smartphone sees the augmented scenes, as this might be the ultimate goal, but I’m confident it could be done today with a turnaround of only a few minutes, which seems to be another acceptable compromise as we wait for the technologies to catch up to our dreams.
I’m looking forward to future chapters!
David
Erica Hargreave says
Oh, that is a clever idea, David, and would create a really fun keepsake from the day, as well as act as great crowdsourced storytelling marketing device for the experience. Will send Nadia a note to make sure she sees your idea too, as it is a smart future addition to the experience.
dreemsteem says
Absolutely gorgeous post!!! Love that this is in DreemPort! hehehe (I might have gotten a sneak peek of it before its release to the dreemers tomorrow. I guess it might have something to do with my name LOL)
You know – I was reading this and just fascinated that they brought this village back to life with AR… but hearing about the pros and cons made me wonder – why not just have it built as a virtual reality piece? I mean – you could still have bonus scenes maybe – heheh that are connected with the actual location!! but… what you said – about the hot sun, and the inability to restart again easily… etc.
I have done some AMAZING “walkthroughs” of locations on the Oculus – and it was just SO much fun in the comfort of my own home!!! hehehe
Either way – VERY cool to see these kinds of things coming together with tech!!!
thanks for sharing!
Erica Hargreave says
While I would create a virtual piece in addition too, there is something about being there in real life and seeing a window into the past, of how things use to be there, that is really cool and awe inspiring. That’s where AR comes in, and what fascinates me about it.
Thanks for reading.
samsmith1971 says
Augmented reality is in its infancy. So many opportunities abound for this to be used far more extensively in education. It can bring almost any subject to life and create a more tangible learning experience for the student. The more senses deployed while learning, the better the understanding and memory retention. I hope they do progress to more *live-action* AR. Creating that sense of presence is a game-changer! Thank you for sharing this. I came across your post because it was promoted on the Dreemport Discord Server.
Erica Hargreave says
Yeah, there is a lot of potential here, as the tools become more accessible and able to manage the large data use of AR.
amberkashif says
It was a good and informative read. Truth be told I never thought of argumented reality being used in visiting the past.
Because the tech is in its warly phases certain problems are sure to occur. However, with the passage of time as there would be more development there will be more sophistication.
Erica Hargreave says
Absolutely. Thanks for reading and sharing your thoughts.