Share
Recent mathematical advances have enabled non-metric cameras and UAV-acquired imagery to be applied in photogrammetry. Now, precise 2D maps and 3D models can be produced using image-processing software that adjusts for images that have larger orientation differences and more unstable camera parameters than those used in traditional photogrammetry. This allows consumer-grade cameras with a variety of lenses to be used for 3D modelling, lowering the entry barrier. â©
Image-processing company Pix4D wanted to test the feasibility and efficiency of merging a variety of indoor, terrestrial, aerial oblique and nadir imagery into a single 3D model. Their project resulted in a comprehensive reconstruction of the Chillon Castle in Switzerland.â©
Several questions they wanted to answer were: â©
- What is the most cost-effective and time-efficient way to create a 3D model, without sacrificing accuracy?â©
- Can various lenses and cameras be used for the same project? If so, what are the advantages and disadvantages of doing this? â©
- Can one create an accurate 3D reconstruction, not only by using different lenses and cameras, but by also merging various terrestrial and aerial images together?â©
The Chillon Castle was chosen for the project because of the challenges presented in mapping it. As an island castle located on the shore of Lake Geneva, its location on the water meant images had to be acquired by land, air and water. The structure consists of 21 independent, yet partly connected, buildings and courtyards. These are difficult to merge into one 3D model, particularly the outside and inside of the structures. â©
To successfully create a comprehensive reconstruction of such a structure, a variety of image types are needed. Nadir images are taken by air, facing down at 180°. Aerial oblique images, usually captured by UAVs, are used to capture information difficult to acquire with nadir imagery. If it is not possible to cover all areas with nadir and aerial oblique imagery, terrestrial images taken with a hand-held camera can be used to fill in uncaptured parts and provide additional information. For most indoor data acquisition, capturing images by hand is sufficient. â©
For the Chillon project, 6,200 images were acquired in one day using four different cameras. Ideally, a reconstruction like this can be done with just two cameras â one on a UAV and a handheld camera (DSLR, ideally with a fisheye lens) â but the project used four to experiment with different accuracies. â©
LIDAR scanning of the entire castle was not feasible for the project. For the complex and indoor areas of the castle, the number of viewpoints needed due to self-occlusions and small corners would run into the hundreds, rendering the project impractical in regard to time. To acquire nadir scans, aerial LIDAR would have been necessary, making the cost of such scanning prohibitive.â©
Nadir and oblique UAV imageryâ©
For the nadir and aerial oblique imagery, two drones were used: the DJI Phantom Vision and a custom quad-copter with GoPro Hero3+ Black Edition camera attached by a GoPro gimbal mount. The budget of the project, the accuracy needed, and the security were all important factors in choosing the UAVs. The DJI Phantom and custom quad-copter were selected because they were lightweight and presented little danger of damaging either people or the castle if they fell. Because part of the mapping was done over a lake, these drones would also present less of an investment loss if, for some reason, they were to fall into the water. â©
The DJI Phantom Vision UAV took most of the aerial oblique imagery and was manned by one person, who mapped all five courtyards in two hours (712 images). The Phantom also captured the street and lake sides of the castle in two hours with two people (469 images). â©
For the nadir and additional aerial oblique imagery, the GoPro and one person acquired 725 images in 30 minutes. â©
Terrestrial imageryâ©
Handheld DSLR cameras were used to acquire the imagery from the interior courtyards as well as the inside of the castle. Two people took these images in three hours with a Canon 6D 8mm fisheye (1885 images) and Sony Alpha7r 8mm fisheye (1933 images). A tripod was used with these cameras, with the settings adjusted to keep the image-capturing stable.â©
For indoor mapping, fisheye lenses were used because of their ability to function in limited spaces and their wide range of focus and field of view. With fisheye lenses, scenes can be captured at close distances and with far fewer images than with a normal lens, making them more time-efficient for indoor modelling. â©
However, fisheye lenses have large distortions and non-uniform ground sampling distances (GSDs), making them less accurate than perspective lenses. â©
The image data from the various cameras was captured with a very high overlap of 80-90%, which is easier to achieve with the use of wide angle or fisheye lenses.â©
Ground control points and tie pointsâ©
The key to merging outdoor and indoor images is the presence of sufficient tie points (either automatic or manual) in the doorways connecting the outside to the inside. High overlap there is essential in ensuring there are sufficient matching points in the images to properly connect the two projects. â©
Establishing ground control points (GCPs) ensures that a project is accurately geo-referenced. GCPs also serve as tie points between nadir, aerial oblique, terrestrial and indoor image projects. â©
As Chillon is a cultural heritage site, placing surveying marks on facades was prohibited and therefore, distinctive natural points of the buildings served as control points. A network of GCP (a tachymetric network) was measured with a Trimble 5601 Total Station from the land side of the castle. Points could be surveyed with reflector-less distance or as spatial intersection without distance measurement (for example, the spheroids on the top of the roofs). The GCPs established needed to be visible in both the terrestrial and aerial images to be recognised and measured in the image-processing software. â©
For the points inaccessible with the total station, a Trimble R10 GNSS receiver was used in RTK-mode with a virtual reference station to measure 11 marked points in two different satellite constellations. The mean difference between the two satellite measurements was 10mm in x and y and 20mm in z. â©
After the least square adjustment in the software, the mean standard deviations of the GCPs were 8mm, 7mm and 6 mm in x, y and z respectively. GNSS-point errors were taken into account and refined before the adjustment.â©
Merging nadir/oblique and aerial/terrestrial imageryâ©
Merging indoor and outdoor, terrestrial and aerial, as well as oblique and nadir imagery together is difficult due to the varying overlap, distance, scale and change in angle between the images. â©
In Pix4Dmapper, you can create one larger project by merging smaller, already computed sub-projects and bringing them into the same coordinate system, so long as any two adjacent sub-projects have more than three points in common. Processing smaller projects separately saves time, as they are faster to process and monitoring individual quality is easier. â©
For Chillon, the images of the various cameras and sources were first processed independently, with each of the 19 sub-projects of approximately 350 images taking around three hours to process. After independent processing, the subprojects were then merged into one project, which took one day of work. At each step of the merging process, a separate bundle adjustment was performed using a re-optimisation option in Pix4Dmapperâs rayCloud editor. â©
The project was considered a success, and showed that not only can several different cameras and lenses be used in the same project, but a very complete 3D reconstruction can be accomplished by taking nadir, aerial oblique, terrestrial, indoor and outdoor images and merging them together.â©
To acquire nadir scans, aerial LIDAR would have been necessary, making the cost of such scanning prohibitiveâ©
Krista Montgomery is a writer at and Christoph Strecha is the CEO and founder of Pix4D (www.pix4d.com)
