Drones are an innovative way to get a comprehensive scan of objects or scenes that are impossible to take from your perspective on the ground. Sketchfab Master Matthew Brennan goes over the types of drones, the basics of flying one, and how to effectively piece together a scan.
My name is Matthew Brennan, and I’m a PhD student in Informatics at Indiana University. My background is in Architectural Design, but I’ve been working as an archaeologist and multimedia designer. I’ve worked on a number of projects that have required the digitization and survey of small, medium, and large-scale landscapes – ranging from a small structure or ruin on a hillock, to an entire terraced archaeological site, or a complete mountaintop. In the past, one might have to survey the site by hand, or rely on dated topographic maps – these methods would produce relatively sparse data of potentially variable accuracy, depending on the skill, rigor, and desired tolerance of the surveyor.
The recent consumer availability of advanced unmanned aerial systems (UAV/UAS), paired with high resolution, lightweight cameras, and increased computing power for photo-modeling means that, presently, for a fraction of the time and money it once took, we are able to create highly accurate models of large structures and landscapes that can be used for survey, AR and VR applications, and real-time display on the web, right here on Sketchfab.com!
Here is a brief overview video of generally what we’ll be covering in this tutorial:
This tutorial will tell you (almost) everything you need to know about creating an accurate 3D model of an object or landscape using a consumer-level UAV.
We’ll cover the following:
- The currently available UAV options priced under $1,000 USD.
- The basics of UAV flight for survey and photo-modeling, as well as some basic rules and regulations to be aware of.
- Processing your photos in Agisoft Photoscan to create a 3D model, and using some additional features in Photoscan Pro to generate orthographic photos.
- Editing the model in Photoscan to optimize it for upload to Sketchfab.
In short – the following image shows the steps we’ll cover:
While certainly not exhaustive – this tutorial will give you the basic knowledge you’ll need at every step from making an informed decision when purchasing, to how to best edit and share the 3D model you’ve created. If you’re already experienced with using UAVs for survey and modeling, you probably won’t learn anything radically new here – this tutorial is written for beginners – but I’ll try to share all the tips and tricks I’ve learned over the past 5 years of flying and photo-modeling!
Before we dive in, here are some example models of objects at different scales, all photographed from an aerial platform, created and edited totally within Photoscan, and uploaded to Sketchfab:
Small Scale: Ancient Obelisk, Boboli Gardens, Florence, Italy (POI [point of interest] photos only)
Medium Scale: First Christian Church and Monroe County Library, Columbus, Indiana (Orthographic and POI photographs)
Large Scale: Cosa Archaeological Site, Tuscany, Italy (Orthographic photos only)
Unmanned Aerial Systems and Vehicles (UAS/UAV):
Unmanned aerials systems and vehicles, or UAS and UAV for short, are commonly referred to as drones. This isn’t particularly accurate, as many of the consumer-level UAVs are not autonomous. In other words, they require a human pilot giving input – they don’t fly and make decisions on their own. Most consumer-level UAVs are quadcopters, meaning that they use 4 propellers for flight. High-end systems can have 6 to 8 propellers, allowing them to carry larger cameras.
Presently, there are a number of relatively inexpensive “flying cameras” available to consumers – but not all of them are suitable for photo-modeling and survey. The key factors to consider when selecting a UAV for these purposes is the resolution of the camera (if the drone comes with one), the availability of a gimbal (for stabilization and orientation of the camera), and/or the payload capacity of the UAV (the maximum weight in camera/lens that the drone can carry and still safely fly). I’ll focus on offerings from 3DR and DJI, two of the major drone manufacturers, as I have the most experience with these systems – but UAVs made by Yuneec are also worth considering.
These are the unmanned aerial systems we’ll examine (prices as of January 2017):
3DR SOLO ($399 USD)
DJI Phantom 3 ($400 USD)
DJI Phantom 4 ($1,000 USD)
DJI Mavic Pro ($999 USD)
Each of these quadcopters has different strengths, weaknesses, and features. What follows is a brief description of each, with a breakdown of the features that concern us for photo-modeling and survey.
The 3DR SOLO can be found at greatly reduced prices recently and is, overall, a great buy. It offers out-of-the-box live video on a tablet or smartphone, and includes a camera gimbal designed to work with the GoPro Hero 3 and Hero 4 (not included with the UAV).
Key Points for the 3DR SOLO:
– Live video from GoPro, but camera is not included
– Gimbal for GoPro 3 and 4
– “Open-source” design allows for mounting of 3rd-party accessories and cameras
– “Smart Shots” such as “Orbit” that can be helpful in photo-modeling
The DJI Phantom 3 is an older version of the Phantom series that comes with a 12MP camera and gimbal. It is a basic quadcopter that offers live video downlink via the DJI app to a smartphone or tablet.
Key Points for the DJI Phantom 3:
- 12MP non-fisheye lens
- Camera and gimbal are included
The DJI Phantom 4 and Mavic Pro are more recent offerings from DJI. They both come with cameras, gimbals, increased flight time and range, and sensors that can help prevent crashes. The Mavic Pro is foldable and compact, making it ideal for travel.
Key Points for the DJI Phantom 4:
- 12MP non-fisheye lens
- Camera and gimbal are included
- “Vision” sensors that detect obstacles
- Live video via tablet or smartphone, and ability to control the camera settings
Key Points for the DJI Mavic Pro:
- Same technology and features as the Phantom 4
- Foldable, compact design makes for an incredibly transportable system
Some major points to keep in mind when using a UAV for photo-modeling is that the camera sensor size, lens, and ability of the camera to be remotely-triggered or set to a timelapse mode are all crucial. All of the above systems meet these requirements. Timelapse mode and/or remote-triggering of the camera shutter are perhaps the most important aspect of a camera used on an aerial platform.
The GoPro cameras, available on the 3DR SOLO and DJI Phantom 2, offer an extremely wide, fisheye lens, ideal if you are photographing something in tight quarters, or if flight height or distance from the object of interest is limited by trees or other obstacles. The wide field of view of the GoPro also helps with orthographic images, allowing the camera to capture the sides of buildings and structures even though the camera is pointed straight down.
The cameras available on the DJI Phantom 4 and Mavic offer more traditional camera lenses – they are wide angle, but not ultra-wide or fisheye, meaning that there is little distortion. For the purposes of photo-modeling, this is sometimes better, although you may be limited by how close to an object you can fly, while still keeping everything in frame.
Worth noting is that there are a number of aftermarket camera mounting systems available for the 3DR SOLO. This means that you can mount infrared, thermal, or mirrorless and micro-4/3rds cameras onto the aerial platform, allowing for a wider variety of lenses and higher-megapixel camera sensors.
Basics of using a UAV for photo-modeling
Photogrammetry, or photo-modeling, is a technology whereby a 3-dimensional digital surface can be reconstructed from a series of photographs. It is often used to create highly accurate 3D models of real-world objects, such as this Ancient Greek bronze sculpture:
Thanks to the availability of inexpensive UAVs, we now have the ability to rapidly and cheaply digitize and survey buildings and landscapes that would have been impossible to do by hand, or prohibitively expensive to do with an airplane or helicopter. However, before ordering a UAV and beginning to fly, there are some important rules, regulations, and etiquette to be aware of. The following regulations regarding the FAA are particular to the United States, so be sure to check with your local administrative equivalent in your home country.
-As of June 2016, all UAVs must be registered with the FAA.
-Commercial UAV operators must pass the Part 107 pilot certification.
-Never fly over 400 feet AGL, or within 5 miles of an airport or helipad.
-Never fly over crowds or sporting events.
-Do not fly at night or out of line-of-sight.
These are just a few of the many rules to be aware of – please check the Know Before You Fly website for a more in-depth explanation.
The best way to think about photomodeling an object or landscape using a UAV, is to imagine how you would approach the object if it were very small, and you were able to photograph it with a handheld camera. For example, in the below diagram illustrating photo capture for photo-modeling, photographs are taken of an object at various heights and angles, in 360 degree coverage.
Capturing imagery from an aerial platform works in much the same way. The camera must either be set to time-lapse mode (a photo is taken every 2 or 5 seconds) or remotely triggered via the UAV controller (a single photo is taken when the pilot presses the capture button). If you are scanning a single “object”, such as a building or column, Point-of-Interest (POI) photographs work best – that is, the camera is pointed at the object of interest, and the surrounding landscape is ignored.
However, if you’re interested in capturing a terrain or geographical feature, Orthographic images are best. Orthographic photographs are where the camera is pointing straight down at the ground, and the aerial platform (the drone) is flown in a grid pattern over the area of interest. It is possible to combine POI photos, Orthographic photos, and even handheld photographs in the same project for the best possible model:
The UAVs listed above can all fly for between 15 and 30 minutes per battery, which is usually enough time to gather at least 200 – 400 photographs of the object or landscape you’ve been flying over. Of course, if you have multiple batteries, it is possible to combine the photographs from multiple overlapping flights, so that even very large areas or objects can be scanned in detail.
For Part 2 of this tutorial, click here