Using Gaming Tools to Make Interactive Digital Heritage

About

My name is Javier Garcia-Lajara and I’ve been working creating art for the video game industry for more than 20 years. During this time, I’ve been involved in many AAA titles like the good old Commandos saga and many others.

Three years ago I switched to Virtual Reality production, first at Future Lighthouse where we created some awarded titles and now with my own production company called Overlat.

I also coordinate the XRLab by Entropy, where we have made some great experiences like “Enter the frame” for Thyssen-Bornemisza Museum where you can literally get into three of the paintings owned by the museum, and VRTogether, an investigation project funded by the European Union about social interaction and self-awareness inside VR experiences.

When I started working with 3D, not many of the tools we have nowadays existed. The evolution of these tools (no tools for automatic UV generation existed in the late nineties!!!) and the emergence of new techniques like PBR rendering, has made everything much easier, satisfying, and accessible, letting you focus on your work instead of finding a way to achieve a certain mood or effect.

I first started as a 3D generalist but after some time I realised that I prefer lighting and shading and environment assembly tasks, and was always looking to test new technologies and features.

Starting with 3D

As a history and technology enthusiast, I’ve always tried to find a way of working with both things, and when I discovered the photogrammetry process, I felt that I had found the perfect mix for it.

From that moment I started to learn the basics of scanning small objects and then I graduated to bigger things using better and different gear, from top grade cameras and lenses to drones and LIDAR.

It is really a pity that all the powerful technologies that were developed by the game industry during all these years, such as real-time engines (Unity/Unreal), shading tools (Allegoritmic/Quixel), and texture streaming solutions, are not well known outside the entertainment industry. I usually try to show good examples of high quality graphics and interaction to break the disbelief for using “video game” tools.

Tools and Workflow

My latest works heavily rely on scanned assets.

I usually do the job with a camera like the Sony A7RII, and different set of lenses depending on which things is going to be scanned. For exteriors I use a mix of 12mm and 24mm, taking advantage of the wide FOV, and for small objects I use a 50mm lens.

The use of LIDAR is often necessary if light conditions are not good enough or the subject is too complex for photogrammetry, as the cave of La Garma where I could only walk through certain areas and was unable to take pictures from every angle.

I also have a drone, Mavic 2 Pro, which is extremely useful for large scenes or ground jobs.

After image acquisition, I make some adjustments in Lightroom in order to get rid of darker areas and adjust the white balance prior to exporting the final JPGs to use in a photogrammetry app. From there, I export some low budget meshes and textures to finally add them to the real-time engine.

Here is where we make the final tweaks for graphics, like lighting, but starts the coding part for adding interaction.

We try to bring the use of real-time render engines like Unreal and Unity to the projects we are working on with other fields, like archaeology, for example, to add new possibilities to the preservation and dissemination of cultural heritage. This means interaction with the content, storytelling to have not only information, but also a story to tell, multi-user experiences, distribution options, etc.

Looking ahead

Something that I want to experiment with is how to be more narrative using services like Sketchfab. Something that lets you surf through 3D content, add multimedia annotations to the content, and then embed it on a web page to view on a mobile device opens up lots of possibilities. It’s a constant source of inspiration and great ideas.

I really believe that cultural heritage should be within reach of every person in the world, wherever they may be, so this service is so useful.