Introduction
One of the sad things about Tokyo is that the city is not precious about its old buildings. Tokyo is in a constant state of flux and it seems like you are never more than a few meters away from a building site. Old and interesting buildings are constantly being torn down. Izakayas, factories, quirky shops, and large family homes being replaced with apartments, malls, and office buildings.
Perhaps it’s because I used to live in the historic city of York, or maybe I’m just getting old, but this loss of history makes me a little sad.
I’m Matt Hudson (@mechpilot) and I am an Augmented Reality (AR) engineer working in the Tokyo office of The Designium. My main role in the company is creating AR prototypes and I often use models from Sketchfab as placeholder assets in my experiments.
One day while browsing Sketchfab, I found the model of an old coffee shop in Akihabara. The building had amazingly survived World War 2 but had not survived the march of progress.
The first prototypes
This discovery suddenly gave me an urge to see the building in its original location. I created a prototype that used Google Cloud Anchors to relocate the photogrammetry scan next to the river in Akihabara.
#AR ghosts of old tokyo ?
restoring an old coffee shop using #GoogleCloudAnchors#madewithunity #designium #AugmentedReality #Photogrammetry #GoogleDevelopers #GoogleCloud pic.twitter.com/8awZ2lcIC0— mechpil0t (@mechpil0t) November 9, 2020
The prototype, while very simple and a little janky, worked well and made me want to try more. To achieve this, I contacted the very talented creator of the scan who seems to share my passion for old buildings: masanaga.
With his permission, I made a number of other AR restoration prototypes using his photogrammetry scans. I used Unity and the visual positioning solution ‘Immersal’ to create various new prototypes for both smartphone and Nreal devices. I called them “Tokyo Time Machine”.
さくらんぼ (Cherry) near Ikegami Station, restored with the #Tokyo Time Machine using #AugmentedReality and amazing #Photogrammetry by @tasklong#madewithunity #designium #AR #history pic.twitter.com/ge0wId1zBd
— mechpil0t (@mechpil0t) April 27, 2021
Sketchfab API
While these prototypes all worked well, they all required that the model be downloaded and imported into Unity before building the application. This limited the scalability of the concept and so I decided to implement a way to download models on-demand using the Sketchfab API.
To begin with, I made sure I could successfully download and import Sketchfab models into Unity, at runtime, on mobile devices. Here, I download and import one of my own photogrammetry scans to be viewed on Nreal.
from browser to #AR ?
a runtime #Sketchfab downloader/importer for #unity3d running on #nreal ?️ #AugmentedReality #designium #Photogrammetry #madewithunity pic.twitter.com/mxMM97SxtK
— mechpil0t (@mechpil0t) April 15, 2021
After this successful test, I then created a system that would automatically load a Sketchfab model based on the location of the user. The locations of the user would be checked against a list of available historical models, and, if the user was close enough to the model, would be downloaded and imported into Unity at runtime.
#sketchfab to #AR version 2
1. GPS is used to find the closest #Immersal map
2. the map ID is used to automatically load the #sketchfab model URL
3. once downloaded the model is automatically aligned#AugmentedReality #designium #Photogrammetry #madewithunity pic.twitter.com/j16DMfERM4— mechpil0t (@mechpil0t) May 1, 2021
This was achieved by creating my own online database that stored the model’s Sketchfab ID (UID) and the GPS coordinates of the model’s location. First, this database was queried to find the closest coordinates, which were then used to make the Sketchfab API call.
For this prototype, I used another location in Akihabara where an historic Izakaya, the Kanda Shokudou Diner, had been recently torn down.
Conclusions
Photogrammetry is the best form of digital preservation we currently have for 3D objects, and in this context, AR can be seen as a form of digital restoration. I feel that the digital preservation of our cities is an important next step in heritage conservation. Hopefully, one day using AR we might be able to access the “version history” of a city and I am grateful that Sketchfab is providing the tools to make this kind of prototyping more streamlined.
At the moment the apps described in this post are internal prototypes and so are not available to the public. However, development is ongoing and I hope to add a map-based search function in the next stage to improve UX and allow users to more easily see what models are available in their area.
About the author
Matt Hudson
AR Engineer at the the Designium
