Share
While geo software is ever more accessible and geo data is a ubiquitous part of the services upon which we now rely, the skills and understanding of the geospatial data scientist have never been more valuable.
Geo location is at the core of some of the digital disruptors that define our age: Airbnb, Uber, Tinder, Rightmove, Just-Eat. The sharing economy and the gig economy are built on combining spatial analysis with broader, bigger data. And, we are just at the start of this change.
It’s not just tech start-ups that are enabling this change: geo-digital innovation is tearing through and revolutionising traditional service delivery in almost every kind of business; from field services, support and maintenance, to delivering more relevant experiences to customers on the move. And as we see the development of the IoT and Industry 4.0, this mash-up of geo with other real-time data and analytics is set to explode in importance.
Broadening the skills base
The career path for geospatial data specialists has been changing for some time. More intuitive software is certainly making traditional geospatial analysis more accessible to the mainstream. However, there is increased emphasis on combining expertise in geo data analysis with secondary skills in the broader data sciences.
And just as increasingly intuitive software has made geospatial analysis accessible to non-specialists, conversely, the same process is happening in data analysis and even software development, with Open Source tools making these areas more accessible to geo specialists looking to build out their skill sets. But as in all areas, choosing, using and specialising in the right technology is vital.
There has been too much hype around the term ‘Big Data’. Scale is just one of the important parameters in modern data analytics. Speed (for real time-monitoring and analysis), data quality and enrichment, and data diversity (combining different sources), and finding relevance are now equally important.
Combining data sets
Modern search technology is about far more than using Google and website search boxes.Underlying this is technology built to find trends and relationships, across multiple, diverse datasets. These can have different formatting or even include unstructured data. Adding in a new data set takes minutes, not hours or days.
Search indexing effectively creates the ultimate flexible structure around the data, allowing the end-user to explore and correlate data from diverse sources. A more rigorous data schema can be added, but it is not a prerequisite. And the indexing process removes much of the legwork.
This ability to freely explore data and add-in entirely new datasets can be incredibly empowering in the hands of a seasoned geo expert. When data are correctly correlated, cross-referenced and enriched, they can be transformed into far more valuable information. With search, it becomes possible to rapidly trial, test and incorporate layer-upon-layer of new data, context and information to any map. In well-informed hands, this power can uncover significant new business benefits.
Real-time views
Speed is also vital for effective geo-digital innovation. Many of the most valuable modern mapping applications are about delivering information and insight based on multiple real-time factors. Not least, where people or ‘things’ are located at any given point in time. The map becomes less a historical guide, and more of a real-time model that is capable of augmenting reality.
The geo data skills remain the same, but the user must harness real-time data technology. Too many first generation ‘Big Data’ tools handle scale, but struggle to deliver real-time analysis without complex customisation.
Within tools such as Elasticsearch, the underlying indexing can be updated in real time. This means ‘live’ deliverables such as heatmaps can be generated in seconds.
Even more than this, the most effective search technologies can process both real-time data and store vast quantities of historical time-series data. This opens up a whole new view, allowing real-time metrics to be viewed in a historical context. It becomes possible to build models and create alerts, even to use Machine Learning to spot anomalies and outliers without knowing exactly what it is you’re searching for.
Distance and relevance
Another of the strengths of search is in defining relevance. Within a search application, the user is able to define their own precise method for ranking the relevancy of any results. This ability to rank different factors in a unique ‘secret sauce’ recipe can, for many end users, be the overriding factor in judging the value of a search application.
In many modern applications, proximity is a key factor in relevance. However, normalising distance against other relevancy factors can be extremely complex. Luckily, this is a common problem which, in Open Source communities, can get solved fast. The most popular Open Source search tools such as Elasticsearch offer ready-made solutions that make it simple to incorporate normalised distance calculations into complex, multi-faceted rankings.
By combining relevancy with speed and scale, search offers an unbeatable combination for building and interacting with complex maps in real time.
Drilling down
This brings us neatly to drill-down. There are many more benefits to using search, but drill-down is perhaps one of the most important for many mapping applications.
As more layers of data and context are added to the information displayed on a map, so the map itself becomes more abstracted from the underlying data. Maps based on real-time data offer an incredible user interface for monitoring and alerting. Multiple complex monitoring tasks can be reduced down to clear, easily understood graphics. But, when alarm bells ring, the ability to drill-down through the abstraction to look at the root causes can become critical.
Here, search technology reveals some of the true power of indexing. As we have discussed, indexing allows near real-time data monitoring and processing. But to do this, it doesn’t cut-down or abstract away from the underlying data. All the original data points are left intact.
This process allows users to drill down through a map to see the underlying data. Maps can be combined with dashboards to reveal all aspects of the data used in their creation. This can be iterated over as many levels as needed. And access to each aspect of the data can be tightly controlled and administered.
It’s all about the user…
In many ways, the map can be seen as the original user interface. Points on a map are now doubtlessly just a part of broader enterprise data. But a clear understanding of geo data, the ability to layer information onto a map, and the need for creating innovative, intuitive user interfaces hasn’t gone away.
Geo is hitting the mainstream in business analysis, internal operations and, increasingly, within customer facing applications. With these applications, the key lies in the ability of end users to extract meaningful information from highly complex data.
Many modern mapping applications put the power of data in the hands of users who don’t always understand the underlying workings. It puts added pressure on the skills of those creating these applications, for while fools cannot create a fool-proof user interface, real experts are capable of safely putting superpowers in the hands of anyone who can read a map.
The challenge for geospatial data scientists is to unlock the value of their expertise for new use cases and transfer the results into the broader business world. There you have it! Choose the technologies that allow experts to make the most of their knowledge and turn this into business value, quickly and easily.
Alex Francoeur is Principal Product Manager for Elastic, a distributed company whose teams, operating in 20 countries, help people explore and analyse their data differently using the power of search.(www.elastic.co)
