Share
Rob Morrison describes how those who have already invested heavily in collecting imagery and remotely sensed data can leverage the power of GIS and Machine Learning to detect otherwise hidden trends and patterns.
The convergence of artificial intelligence, remote sensing and location present massive opportunities to GIS professionals and together they can be utilised to enable effective and prompt decision making.
Derived outputs obtained by combining these three fields can provide valuable insight, allowing patterns to be discerned where none was previously obvious, permitting those who can interpret this data and exploit this potential. It’s “The Science of Where” in action.
So what’s in it for you and why does this matter if you’re in GIS?
The use of remote sensing is important because it contains masses of valuable information if you know how to extract it. This type of imagery is captured on sensors mounted on aircraft, unmanned drones or satellites. Imagery of these types are increasingly used in many areas such as commercial industry, scientific research, emergency and resource management and security and reconnaissance.
This, coupled with the almost ubiquitous availability of drones and the low entry point they provide for obtaining high quality aerial photography, has led to many potential applications and uses that may traditionally have been impossible to achieve for organisations or agencies with smaller budgets.
Imagery + Machine Learning = Rich Information Products
So, let’s step back and have a think about imagery, in its raw form imagery is just raw unstructured data. Binary files with rows and columns of pixels with numbers assigned that mean something to somebody according to a scale.
Imagery contains data – machine learning allows us to extract information from that data. Combining the power of GIS and Machine Learning, we can transform this unstructured data into geospatial information products from which to derive results and actionable information. Applications such as precision agriculture, change detection, target detection and asset management are just four applications that can benefit from consuming these derived products.
Taking a Geographic Approach to Machine Learning
Another such application is detecting building footprints. Late last year my colleague Jonathan Sloan and I held a webinar on ‘Taking a Geographic Approach to Machine Learning’,1 a part of the Esri Ireland ‘Do One thing Well’ webinar series.2 In that webinar we utilised machine learning techniques and orthophotograph’s to try and detect building footprints.
The process we used was achieved using ArcGIS Pro 2.4.2 and the Out-of-the-box installations of Anaconda and Jupyter notebooks that come with it.
Our approach in seven simple steps
1. Select a training image
Our aim was to determine how effective an automated approach would be when compared against a known output. The imagery used was a small sample area of 3 Band, 16cm resolution Orthophotography covering a small area in Bangor, Northern Ireland captured in 2016. (Fig.2 above)
2. Identify Training data
For the input area we also obtained polygons for existing building footprints from Land and Property Services. This was our baseline dataset that we would use for training and for comparison against our machine learning output. (Fig.3 above)
3. Create a Label dataset
The first step in the image classification workflow was the creation of training samples. Training samples “teach” the deep learning model what the size, shape, and spectral signature of a building footprint may be. The more samples provided to the model, the more accurate the result will be. We used the LPS Building Footprint polygons to train our model and from this created a simple, 8-bit unsigned label dataset or classified raster image as shown in Fig.4 below.
4. Training the model
This is the dataset we can use to train our model, using this as the Classified Raster input to the ‘Export Training Data for Deep Learning’ Geoprocessing Tool along with our orthophotograph as the Input raster, we can set some basic parameters and then export image chips which can be used to train our deep learning model.
The output of the training is an .emd file which is an Esri Model Definition file. This file is the bridge between the GIS and the deep learning framework.
5. Inference
Now that we have a training dataset and have created our model, we can run that model against a different set of orthophotography, in our case we used 16cm 3 band imagery of Belfast captured in 2018. We do this by running the ‘Classify Pixels using Deep Learning’ Geoprocessing tool using our Belfast raster and our deep learning model definition created earlier as inputs.
The output of this process is another classified raster, one where the deep learning model has identified clusters of pixels which represent building footprints (see Fig.5).
6. Create Building Polygons
Using this classified raster, we can then run the ‘Raster to Polygon’ geoprocessing tool to convert our raster buildings to polygons (Fig.6).
7. Regularise the output Polygons
This creates polygons with many vertices and of irregular shape so at this point we may wish to clean these up somewhat by running the ‘Regularize Building Footprints’ geoprocessing tool which will remove those excess vertices and square off our building edges. The output of this tool are the green footprints as shown in Fig.7;
As you can see, the process has been largely successful in identifying buildings using an automated approach. Results could have been improved by further refining our training sets and our deep learning model however, for the purposes of our demo we deemed the results to be a success.
I hope that this has demonstrated how Image classification tools can enable you to utilize data collected from sensors and automate the processing of this data with the powerful computational capabilities within ArcGIS to identify and quantify features. You can follow the exact steps we took by watching a recording of the webinar.3
Summary
The simple case study outlined in this post merely scratches the surface of what can be achieved by utilising machine learning and GIS. Imagery and remotely sensed data are a valuable resource for many organisations who have made substantial investment obtaining the data and, who are now deriving substantial value from that data by utilising innovative data analysis techniques.
One such example, presented at an Esri User Conference, describes how an insurance company is using similar techniques to improve its response in times of disaster.
The field of machine learning is both broad and deep and is constantly evolving. Using ArcGIS and machine learning allows organisations to derive valuable new content. With ArcGIS being an Open, interoperable platform, it allows for the integration of complementary methods and techniques that empower ArcGIS users to solve complex, real-world problems in a fundamentally spatial way.
By combining powerful built-in Image analysis tools with any machine learning package users can benefit from the spatial validation, geo-enrichment, and visualisation of results.
Finally, I hope this has sparked an interest in learning how ML and AI could benefit you and your organisation. If you wish to know more about the exciting world of Machine Learning and Artificial Intelligence, Esri provide a wealth of resources to get you started and you could begin by taking a look at the publication to be found here.4
References
Rob Morrison is Principal Consultant at Esri Ireland and has over 30 years of experience in the field of GIS, 17 of which have been working with the ArcGIS platform. Originally from New Zealand, Rob has worked for Esri Ireland for the past 10 years after extended periods working in the UK and the Middle East.
