Our Cultural institutions Page highlights our ongoing support of museums and cultural institutions with free accounts and access to tools. In Cultural Heritage Spotlight, we’ll explore museums and cultural institutions who are using 3D technology to bring new life to their collections. Today’s blogpost article will help you get started with photogrammetry.
The Royal Armoury, Skokloster Castle and The Hallwyl Museum are three Swedish museums that together constitute a government agency, accountable to the Swedish Ministry of Culture. Museums employ about 60 people and received just over 800.000 visitors during 2016.
“Openness and accessibility are core values in our organization” Says Erik Lernestål, Deputy Head of Department, Collections and Digital Resources.
“Our collection database has been open to the public for many years and since 2013 we are also sharing all of our hires 2D images under open licenses through Wikimedia Commons and Europeana. By the end of 2016 there were about 46.000 images shared and we recently passed 50 million page views on Wikimedia.
In 2015, Europeana wrote a case study called “Making a big impact on a small budget” about our work on opening up our digital collections.”
In 2016 we ran a project together with the Swedish Exhibition Agency with the purpose of evaluating photogrammetry as a potential way of making our collections more accessible. We had no prior experience with photogrammetry, or any other 3D techniques, but we had both a photographer (myself) and knowledge about computer software/hardware (Fredrik Andersson) in house. The goal was to create hires 3D models and textures in which one could zoom in to see fine details but still be able to view them online on a normal laptop. Furthermore the quality had to be good enough for our conservators and curators to see it as a potential new tool in their work.
During the project we produced the two following films in hope to help other museums interested in the technique.” (Subtitles in English)
“Digitization in 3D might seem complicated and inaccessible to many museums with small resources, but it doesn’t have to be! Our aim is to share our experiences openly and allow other museums to learn and be inspired to take their own steps towards openGLAM.
We are constantly exploring new ways to further enrich our digital collections with hope to make the cultural heritage within our care even more accessible. In the physical space of the museums for one, but also and maybe more importantly, online. Because most people will never visit our museums IRL.
Sharing is caring and cultural heritage belongs to everyone everywhere!”
Photogrammetry for beginners
Fredrik Andersson, Digital Coordinator, Collections and Digital Resources, wrote the following article on what the three museums have learnt from their first experiences with photogrammetry:
“Since spring 2016, we have experimented with digitizing 3D objects using a technology called photogrammetry. The technology itself is not new, but its application within the museum sector (in Sweden) is rather uncommon. Photogrammetry is a way of producing 3D models of an object using calculations based on a number of two-dimensional still images.
Background
It was during a conference about digital heritage in 2015 that we first encountered photogrammetry. We decided to test the technology ourselves. We had previously experimented with 3D laser scanning, with the assistance of external expertise. With photogrammetry, we saw the opportunity to achieve good results on our own.
Our main purpose for testing photogrammetry was to explore new ways of making our museums collections more accessible and we now see a vast potential for the technology. The photogrammetry experiments have been run as a pilot project where we collaborated with The Swedish Exhibition Agency. In April 2017 we will do a joint presentation at the Annual national conference for Swedish museums and give a workshop on the technology.
Applications
Photogrammetry can be useful in many ways. For documentation as well as for interaction with the public. In a museum exhibition, a 3D model can allow for showing angles and details that would not otherwise be exposed. A 3D model can be a complement to 2D photos or video clips for users who do not have the opportunity to visit the museum in person. 3D models can also be used as a method of making objects accessible that are otherwise too fragile for being exhibited or handled. Photogrammetry also opens up for different types of interactive applications, where the user can twist and turn, zoom and interact with the objects in online and mobile applications.
For the museums internal work, photogrammetry has many potential areas of use. It could be used by conservators for documentation of objects on loan, creating a better understanding for an object’s condition and possible damage. Photogrammetry could also be a great pedagogical tool when giving lectures.
Procedures
We started this work as complete beginners. We did however have solid knowledge of photography and digital technology. We started out as most people nowadays – we Googled.
Not surprisingly, there are plenty of tutorials, beginner’s guides and FAQ:s on photogrammetry out there. We read different instructions for the method of photography for photogrammetry, and then began to delve into the technical aspects of software and hardware for 3D calculations. After a short study period we started with trial-and-error – experimenting and evaluating.
Below is a description of the process, both in terms of photography but also processing on the computer. I want to stress the fact that this should not be seen as a set of rules; it’s simply an account for our method. We think we have achieved a very good result, but we are fully aware that there are many other solutions and approaches.
Here are the necessary assets for the work:
- Enough pictures of the object taken from different angles (more on this below)
- A reasonably powerful computer that can do the calculations required (there’s also cloud-based options)
- A computer software that can interpret and process the images into a three-dimensional representation
Photographing
To get as good results as possible, it is important to be systematic when it comes to shooting the images. The object should be photographed from several different heights and on each level, the object (or the camera) is rotated between each image, so that the photo shoot includes the entire surface structure of the object.
In our tests, we used a motorized turntable on which the object was placed. This accessory is not necessary, but it helps considerably for achieving accuracy in terms of rotating the exact amount of degrees between each image. The camera was mounted on a tripod, where the camera only needs to be adjusted height wise (five different height positions). The turntable rotates the object ten degrees between each exposure, resulting in a total of 180 images per item. The lighting is done with a large and flat light where the objective is to create such an even light as possible, so that no shadows and contrasts affect the calculation of the object’s structure at a later stage in the process.
Computer calculations
Still images from the shoot make the basic material for the 3D model, but photogrammetry also requires a lot of computer processing. In this project, we quickly discovered that there are a plentitude of different software for photogrammetry – both commercial products and free alternatives. Furthermore, there are software applications that cover all aspects of the computer processing stages, but also solutions which use several different applications for the various steps to create a 3D model.
We started to experiment with the application Autodesk Memento (now renamed to Autodesk Remake), but later switched to Agisoft PhotoScan and RealityCapture. The three software solutions we tested are commercial products capable of performing all the various calculation steps needed to produce a complete 3D model. The steps performed in the calculation are as follows:
- Align photos – all photos are adjusted and adapted so that common points, viewed three-dimensionally, between adjacent images is identified – in this step the camera position in relation to the object is also calculated
- Build dense cloud – in this step the identified pixels in the step above are calculated together in an X, Y and Z positioning within the three dimensional space
- Build mesh – In this step, the different points are calculated together into a complex mass of triangular shapes (polygons)
- Build texture – In this final step the two-dimensional images are used to create a photographic area of the model itself, so that the 3D model gets its realistic texture surface
The resolution of the images, the number of images and how complicated the image content is for each image, affects how much time is required for the calculations of the steps mentioned above. During our project we’ve done tests on creating 3D models with several different quality parameters and on different computers – from a powerful laptop to a very powerful desktop PC (with components corresponding to a very powerful gaming computer).
The various calculation steps require different types of resources from the computer, which led to our conclusion that you need to balance the computer in question in terms of graphic card, memory, cpu and hard drive. All components are required in different ways in the various steps, so it’s not enough to have, for example, a very powerful graphics card if the other components are slow or inadequate.
Initially we made the first tests with a powerful laptop, this proved to be completely untenable when the calculation of the first 3D model took over three days to complete… The computer in question had then been calculating at 100% cpu usage and the laptop’s cooling was nowhere near helpful.
Next, we tested two fairly powerful desktop PCs, both in single usage, and also together with a network solution where the two computers did the calculations simultaneously. During these tests, we studied the log files of the applications to identify weaknesses in the hardware setups. Due to the fact that the images we used for the tests had very high resolutions (each image was 50 megapixels), the tests took a very long time even with powerful computers (somewhere around 15-20 hours for each model).
Our desire was to find a hardware setup that could do the job within 8-10 hours when running Agisoft Photoscan. To make this possible we invested in a new computer, where we chose the optimal components based on our previous tests.
One of the other applications we tested (Autodesk Remake) also provides a form of cloud solution where the calculation is done by computers in a cloud service. This can be a very good option if you don’t have the resources to invest in a powerful computer. Although when we tried the cloud service the waiting time was still very long. We had to wait for a couple of days for the final 3D model. Hopefully this was just a temporary resource issue – we tested the cloud service when the software was in a beta version.
Our ambition with the project has been to produce 3D models with very high resolution textures where the sharpness, clarity and fine details of the object is maintained even at close-up zoom levels. As for equipment required, it’s as simple as the better the camera (and optics) used, the better results can be achieved. The question is simply what is adequate in terms of the final 3D model. For example it is quite possible to use a mobile phone to photograph an object with a number of images, from different angles and then create a decent 3D model using an app or computer software. During our tests, we have experimented with objects of different types of materials, and it’s important to mention that photogrammetry is not applicable to all types of materials, for example, you can’t use photogrammetry to create 3D models of objects made of glass.
Completed 3D model and strategy for publishing
We are very pleased with the 3D model of the helmet with grotesque visor. The helmet’s elaborate decoration is represented very well even on closer zooming, and all the little nuances in the surface structure are reproduced clearly in 3D.
Our approach to make digital material accessible is that we primarily choose to use existing and widely used web platforms. Regarding our first 3D models we have chosen to publish them on Sketchfab. Here there are lots of 3D models of various kinds, and the platform includes several useful features in terms of lighting and annotation on the model. 3D models made with photogrammetry are faded and dull in appearance – without any lightning settings the object doesn’t look realistic. But with Sketchfab built-in features it is possible to recreate various types of lighting conditions, allowing, for example, metal to be reproduced with clarity and luster.
As for the helmet we embedded the Sketchfab viewer on our website to make a simple digital exhibit. In addition, we also put up an article on Swedish Wikipedia with a lot of text material about the helmet. We believe that using and combining different kinds of platforms is a good way of making the 3D content as accessible as possible.
Continuation
We have now been working with photogrammetry for about a year, and we feel that we have reached the goal in terms of our quality objectives. Right now we are thinking about how we should proceed strategically with photogrammetry; our aim is to implement the technology as a digitizing method among others in our digitization workflow. There’s some work remaining when it comes to identifying practices and standards in terms of archiving and metadata of the various file formats that 3D models consist of. The file formats are an issue that we are still investigating, where we are keen to find open and long lasting standards.
For the past few months we have primarily been using another application called RealityCapture. This software is able to perform the calculations a lot faster than the other solutions we’ve tried although maintaining a very high quality. We’re still evaluating the possibilities and shortcomings of the different software solutions available.
Finally, we hope that our experiences with photogrammetry can inspire others to try the technology!”
Thank you very much Erik and Fredrik for this helpful and very interesting article.
If you are interested in scanning your 3D artifacts and join our community, get in touch with us at museums@sketchfab.com to set up your free business account.
About the author
Guillaume Deniau
3D for everyone
