About and Background
Hi, I’m Håkan Thorén and I have been working for the Vasa museum in Stockholm for about two years. Thirty years ago I started to work with cultural heritage as an archaeologist. In the beginning it was mostly digging at rescue excavations but during the last 20 years I have been working with technical applications in archaeology like GIS, software development, total station surveying, laser scanning. When SFM/photogrammetry became easy available around 2010 I had to try that as well.
Organisation
The Vasa museum is one of four museums that together form a public agency under the Ministry of Culture called the Swedish Maritime Museums. The other three museums are the Naval Museum, the Maritime Museum and the Swedish Railway Museum. The Vasa museum in Stockholm is the most visited museum in Scandinavia and shows Vasa, the ship that capsized and sank in Stockholm in 1628 on its maiden voyage. After 333 years on the seabed the mighty warship was salvaged. Today Vasa is the world’s only preserved 17th century ship. During the salvage many artefacts were found in the mud around and under the ship. After the salvage there was also a large-scale excavation on board where about 20 000 artefacts were found on the four largely undisturbed decks. Conservation began after the excavation, and the ship was sprayed and impregnated with PolyEthylene Glycol (PEG) for 17 years. Since 1990 the ship has been on display in the Vasa Museum, which had 1.5 million visitors last year.
3D Management Tool
There is a lot of information connected to the ship, not only information about finds, but also information about conservation activities, samples, photographs, changing of bolts, deformations, etc. The staff at the Collections unit of the Vasa Museum have for a long time had plans for a management tool that could host the information and be able to visualize it in three dimensions together with a photorealistic 3D model. After the turn of the millennium the search for a software that could satisfy their needs began. The ship was laser scanned in 2003 but it was at that time hard to find software that could both visualize the ship and display information from a database at the same time. The management tool idea was shelved until recently. Two years ago the plans for a management tool was brought to life again. The Collections unit at the Vasa Museum asked me if I wanted to join them in their hunt for a management tool. It was an offer I couldn’t refuse. However, after searching for some time we still couldn’t find any software on the market that suited our needs.
We sat down thinking and realized that what we needed was more or less a 3D-GIS that was easy to use and that could link lot of different kinds of information together. It was also necessary that our new tool could meet the needs of digitization. Of course it was possible to use a standard GIS package with 3D-capability like ArcGIS but the solution wouldn’t be user friendly and a relational database is not ideal for linking a lot of different kinds of information. Besides, we wanted to be able to view database information together with hierarchical tiled models of the ship. A hierarchical tiled model is a model that is stored with different resolutions and enables us to show a model with a very high level in detail without slowing down the computer. In the end we decided to use the Intrasis software and extend its present capabilities with a WebGL viewer. A WebGL viewer would make it possible to view and digitize data in 3D and also allow us to view models with hierarchical tiles. Intrasis is the most used GIS among archaeologists in Scandinavia and its data model is made especially for storing and relating a lot of different kinds of information. The 3D-capability is currently under development.
Modelling a 17th Century Ship
For the 3D-model the original plan was to use the old laser scan of the ship, create a meshed model out of it and complete it with a texture made by photogrammetry. After a closer examination of the scanned point cloud we decided that it had bad coverage in hard to reach places. Even if modern laser scanners are small, a camera is more convenient than a scanner in narrow cramped spaces so we decided to use photogrammetry as the main technique for our 3D-model.
Photogrammetry may be easy but now the object was a 17th century ship almost 70 meters long and with 5 different decks. It’s also dark aboard and a lot of the wooden surfaces are shiny due to the PEG conservation. Our aim was also to try to mask all modern installations like stabilizing support, climate equipment, electrical installations, fire protection equipment and so on. The poor light conditions aboard made us use an additional light source. It’s always difficult to use movable light sources when taking photographs for photogrammetry. If the light sources introduce shadows in the photos the software that processes the images could interpret shadows as geometry.
Inspired by a Norwegian colleague, Gunnar Holmstad at Nordnorsk Fartøyvernsenter og Båtmuseum, we tested a ring flash. A ring flash is a circular flash that fits around the camera lens and lights up the motif with a minimum of visible shadows in the resulting photograph. The flash was great and lit up the motif without any visible shadows but it gave a lot of reflections in the shiny PEG surface. After attaching a polarizing film to the flash and a polarizing filter to the camera it was possible to filter almost all reflections from the flash, but there was a drawback. The polarizing filter on the camera filtered so much light that the autofocus of the camera had problems finding focus. The ring flash had an inbuilt LED-light but it wasn’t enough to make the camera be able to focus automatically, so we also added a powerful LED headlamp to the setup. The light from the headlamp didn’t affect the final photograph it just gave enough light to make the camera to be able to autofocus. So far we have taken about 17,000 photographs in the upper gun deck and in ⅓ of the hold with this setup.
Photogrammetry workflow
With a big ship like Vasa it’s of course necessary to divide the model into smaller parts. Including the upper deck there are five different decks, a rig and a hull that have to be modelled. Each of the decks is too big to model in one chunk so the model has to be divided into smaller parts. An old wooden ship is usually divided from the bow to the stern by deck beams. The upper gun deck of Vasa is divided into 28 parts by beams. Each part is about 1.1 to 2.6 meters wide and when divided into a port and starboard side we got 56 smaller parts that were more suited for modelling. By dividing the decks into smaller parts and model those separately with high resolution it’s later easy for us to put together models of desired size.
The workflow for creating the model of the upper gun deck:
- Attach markers to the ship and survey them with a total station
- Take photographs
- Correct exposure, rename image files, IPTC-tag images and convert to DNG-format
- Align images
- Detect markers and import surveyed coordinates as reference
- Optimize camera alignment
- Build dense point cloud
- Build mesh
- Export mesh and import back again after cleaning
- Create masks by the cleaned mesh
- Finalizing image masks by manually masking modern features
- Build a new dense cloud with masks applied
- Merge the small modelled parts to a large point cloud and create a clean mesh.
- Mesh editing (decimate, close holes etc.)
- Build texture
Shooting and Georeferencing
Our workflow begins with markers. To be able to easily put our small models of the ship together we use georeferenced markers. Since we use Agisoft Photoscan as software and need a lot of markers we use their 20-bit self-detecting markers. We try to keep them small, about 20 mm, so they don’t interfere too much with the motif. The markers are later surveyed by total station. The camera we use is a Nikon D750 with a 20 mm Nikon lens. To the camera we have attached the earlier described setup with a Godox Wistro AR400 ring flash, polarizing film and polarizing filter. We use manual camera settings, shutter 1/200 and aperture 8 together with the full effect of the flash. After finishing the photographs for one part we rename the image files with the Multi-Rename tool in Total commander. Renaming is necessary so that we don’t end up with a lot of images with the same name. The new name of the image file reflects the part of the ship. The same information is also tagged into the image file as IPTC metadata in Adobe Bridge. If it’s needed we also make minor corrections to the exposure in Adobe Lightroom. It is faster to make corrections after photos have been shot than to adjust the power of the flash or the settings of the camera when shooting.
Image Align
We usually process the images as fast as possible to check that we have covered every part and that we are getting a good mesh. The processing starts with photo alignment. The alignment is done with default settings and is straightforward if we have been careful when taking the photographs. We usually batch process the alignment and the marker detecting. Photoscan is very good and accurate at detecting markers in the photographs and the automatic detection usually finds enough projections to be able to georeference the model. The reference coordinates are imported from a text file that have been exported from the total station and the mean error in the georeferencing is usually less than 1.5 mm. After applying the reference coordinates for georeferencing it’s possible to optimize the camera alignment.
Dense Cloud and Mesh
For the dense point clouds we use low quality settings. If we use higher settings we would drown in points when we put the point clouds of the small models together. We also use mild filtering since we want to be able to see as much of the tree structure as possible. High details of the tree surface is important since it shows cracks and how the water has affected the oak during more than 300 years under the surface. After finishing the dense point cloud we build the mesh. Photoscan is very good at continuous geometry but not so good at discontinuous geometry. This can be seen in the model of the upper gun deck where Photoscan has problems to mesh the gratings amidships. We will try to solve this by using either a laser scanner or trying different software. After meshing we usually export the mesh for “cleaning”. We have tried different software for editing meshes and for the moment we use Zbrush since it can handle high resolution meshes with as much as ten million faces.
Cleaning
The cleaning is important for the masking that is the next step of the workflow. During the cleaning we delete all parts of the mesh that show other parts of the ship than those we are currently interested in. We also delete parts of the mesh that obviously show modern installations. The cleaned mesh is then imported back into Photoscan to replace the original mesh of the model. The new mesh is now used to create masks by importing them from the model. This ability saves us a lot of time since we have found out that using the flash can cause problems when meshing. Since distant parts of the motif get darker than close parts, Photoscan tries to extract information out of almost black parts of the images and sometimes messes up the mesh. This can be fixed by applying masks to the images and ensures that we get a good mesh and texture. The first version of the image masks is later supplemented by manual masking that aims to take away all modern features.
Merging Models
When the masking is finished it is possible to build the final version of a small part model. A new dense cloud is built with the masks applied and after that it is possible to merge the small model point clouds into a large one and build big models with the desired resolution. A bit of editing could again be necessary after building the mesh. The masking can sometimes result in holes in the mesh that need to be closed. If the resolution of the resulting mesh is too high there could also be a need for decimation to adjust the number of faces. The modelling workflow ends with building of the texture.
Future Plans
The management tool is at the moment under development and is planned to be tested and ready for use in the near future. The completion of the 3D model will take a little bit longer. According to plan we will have a complete model of Vasa by the end of 2021 and we plan to continuously upload all parts to Sketchfab. At the same time that we are working with the model we plan to start importing information to the database of the management tool and we will gradually be able to start using it. Probably it is the artifacts from the excavation and information about conservation activities that will find its way into the database first. After that it will for instance be possible to visualize distribution of artefacts in 3D.
Why 3D?
In our organisation we have recently started using 3D models. The model of Vasa for the management tool is one example. Another example is the wreck models that are made by our marine archaeologist at the Maritime Museum.
In the case of the management tool it wouldn’t be possible to visualize the distribution of artefacts without a 3D model, it would only be a pointless swarm of spheres. The situation would be the same for any other kind of information connected to the ship so it is very easy to argue for the use of 3D in these contexts. Today the public is not allowed to get on board the ship and at this time the 3D model of Vasa is primarily for use in the management tool but it would be a missed opportunity not to use it for other purposes. Who knows, in the future it may be possible to get on board the virtual Vasa.
The use of 3D to document wrecks is also a good example of using the technique. Underwater photogrammetry is a lot harder than terrestrial photogrammetry. Depending on visibility conditions you usually have to take more images and the processing software often has problems aligning all images. Despite all these problems, the result can be very good and can give the viewer a much better understanding than from a couple of photos.
Fundamental for the use of 3D-models is the ability to share them. Not so long ago you needed to install different plugins to your browser to able to view 3D content on the internet. Nowadays almost all browsers support WebGL that makes it possible to render 3D graphics but it is more to it. WebGL is only an API and to be able to use it you need programming. This is why we use Sketchfab. Sketchfab gives us possibilities to upload models and easily share them without any coding. Sketchfab’s user interface is very intuitive and the terms of use are quite generous. We also like the ability to “polish” our models in Sketchfab. We usually do some lighting- and colour corrections before we make the models public and we will definitely continue to upload our models on Sketchfab in the future.
Other Great Historical Models on Sketchfab
When I get the opportunity, I have to promote the models from a neighbour museum in Stockholm. The Hallwyl museum started to model the rooms of the museum about a year ago. They have a completely different approach when taking photographs for photogrammetry than we have at the Vasa museum. Since they can use the natural light in the rooms and are very skilled photographers their models has an outstanding atmosphere.
Vasa Museum website / Vasa Museum Instagram / Swedish National Maritime Museums website
About the author
Håkan Thorén
Vasa Museum
