Use Cases 24 Feb 2022

Siren geospatial investigations: An overview of the Enhanced Coordinate Map component

Author: Manu Agarwal
Author Manu Agarwal
Siren geospatial investigations: An overview of the Enhanced Coordinate Map component

Geospatial data analysis is critical across all sectors of investigations, from Law Enforcement to National Security, Cybersecurity and more.

Siren provides multiple components that have mapping capabilities, namely the Enhanced Coordinate Map (a visualization that can be added to any dashboard) and the Map Mode of the Graph Browser.

This post will introduce the Siren Enhanced Coordinate Map (ECM). We will explore some of the key features of this visualisation including the loading of GeoJSON, configuration of the default Base Layer.  We will also provide insight for configuring and working with layers in ECM and how dashboards can be filtered by creating geo filters from ECM.

There are various sources for data on ECM. At the core, it displays data from indices (stored in the backend Elasticsearch) linked to Entity Tables. Additionally, you can ingest overlay and point data that contain spatial information (Stored Layers), from tile servers (WMS/WMTS) or from Map Services (WFS). 

The ECM visualisation can then be added to dashboards, where they become fully interactive visualisations, allowing users to geographically view and filter data, or even add new data by simply dragging other dashboards onto the map. 

The ECM by default displays an Aggregation layer and a Geo Filter layer. The Aggregation layer is based on the configured main search for the ECM visualisation and is created by aggregating geographical data in real time by high performance backend functions like Elasticsearch geohash aggregation bucket. The Geo Filters overlay is a visual representation of geographical filters which have been created directly by users on that ECM.

Geo filters added to the dashboard are applied to all other present visualisations. They also apply when passed to another dashboard via the Relational Navigator visualisation. An option can be configured to toggle whether dashboard filters are applied to Point Of Interest Layers. Geo filters do not apply to layers from WFS or Stored Layer sources.  However, these layers are useful as a way to provide context about other relevant information during the investigation. Examples of all of the above will be provided, so keep reading!

Additional tile server layers can also be configured to allow for quick toggling between the default Base Layer and others. For example, roads, satellite views, or a hybrid from sources such as Google Maps, Bing Maps, or OpenStreetMap.

Configuring the Map with Tile Server in Siren:

Let’s start by configuring the default Base Layer. The default Base Layer will be visible even if your map contains no other configuration. We will use the ESRI World Street Map as an example. Of course, you can configure Siren Investigate to use your own tile service depending on your requirement. You can also use existing free or paid tilemap providers, or build your own Tile Server. 

Note – if your tile server supports WMS/WMTS, it is compatible with Siren Investigate.

The tilemap settings for configuring ESRI World Street Map as the default Base Layer are provided below. This can be pasted directly into the investigate.yml file:

tilemap:
  url: 'https://server.arcgisonline.com/ArcGIS/rest/services/NatGeo_World_Map/MapServer/tile/{z}/{y}/{x}'
  options:
    attribution: 'Tiles © Esri — National Geographic, Esri, DeLorme, NAVTEQ, UNEP-WCMC, USGS, NASA, ESA, METI, NRCAN, GEBCO, NOAA, iPC'
    subdomains: ['a']
    minZoom: 0
    maxZoom: 16
ESRI World Street Map configured as default Base Layer rendered on a configured ECM to gauge the location of data from the Aggregation layer

Aggregation Layer Tooltips:

Metric and visualisation tooltip types are configurable. Metrics provide summary statistics based on the count of documents or any of the average, sum, min, max or unique count based on a specified numerical field. Most visualisations that are configured to the same index as the ECM can be added as ECM tooltips. The data within these visualisations correspond to just the data within the Aggregation marker (i.e. the geohash aggregation bucket) hovered on.

Companies that have coordinates within the geohash bucket of New York shown on an analytic table as a visualisation tooltip

Stored Layers:

There are two provided methods for loading GeoJSON files into Elasticsearch as Stored Layers. These are Folder Structure and the Spatial Path. Both will accept files of type JSON or GeoJSON. The methods differ in how their spatial_path is determined, but once they have been ingested into Elasticsearch, they function in the same way. The most suitable approach depends on the form of your current data.

Note – Custom scripts can be produced for ingesting data as Stored Layers in Investigate. As long as an index contains a geo field type (geo_point for points or geo_shape for lines or polygons) and is prefixed with .map__, it will be treated as a Stored Layer

We will be using the Folder Structure because our GeoJSON data does not have a spatial_path attribute within the properties object of each GeoJSON feature.

Load GeoJSON into Elasticsearch:

Folder Structure
bin/load_map_reference_indices.sh --help
/bin/load_map_reference_indices.sh -p "<path to your GeoJSON folder>/geojson/" --debug --structure --overwrite 

The -p argument is the path to the GeoJSON folder and is required. Other argument details are described in the documentation here. The console output below is a successful load (with –debug argument).

Loading of GeoJSON into Elasticsearch

Visualise the Stored Layers in ECM:

The Layer Control is located in the top-right corner of any ECM. When clicked on, it allows you to select which Stored Layers (i.e. from the example above, the GeoJSONs that we imported) can be added to the map. Select the ‘Populated cities’ and ‘World Countries’ Stored layer and click ‘Add and Display’. This will add it to the map and make it visible. If you are loading many layers, the ‘Add’ option might be useful, as rendering layers on the map can take time.

Add Layers onto the Map

Point of Interest Layers:

Point of Interest (POI) layers are useful for representing other searches (that contain a geo_point or geo_shape type) on the map. In the example below, we are using the companies index to represent a Point of Interest layer. The ‘apply filters’ option is checked which means that filters from our dashboard will also filter this POI layer.

Point of Interest layer and its configuration represented on the ECM

It is also possible to create POIs by dragging a dashboard which has a main search containing a geo_point field.

POIs by dragging a dashboard on MAP

Marker clustering:

Marker clustering is used for point layers coming from either POI or Stored Layer sources. It allows all points on that particular layer to be represented at once.

The image below shows the Populated places Stored Layer on the Map. As the layer contains many points that would overlap at the current zoom level, it is showing the areas where there are higher densities of points as Marker clusters with a number on the cluster representing the amount of points:

Populated places represented as a Marker Clustered POI Layer

In the areas where there are lower point densities, but still contain overlapping points,  grouping happens. You know they are grouped because there is a + to the right of the marker. These can be ‘exploded’ by clicking on them, which is also known as Spiderfying.

Spiderfying grouped points

Layer Ordering:

Layers are drawn on the map in the same order they are in the Layer Control (points always show in front of other layers). Layers can be ordered by clicking the drag handle to the left of any checkbox and dragging them to the desired position.

Layer Ordering and Populated places Stored Layer when added to the map

Geo filters:

Geo filters can be used to spatially filter the Aggregation layer or POI layers (that have the ‘apply filters’ option checked). When on dashboards, other visualisations are also filtered by geo filters. 

They can be created by either using the selection tools or by clicking polygons rendered on the map. 

Selection tools:

An example of the selection tools approach is shown below where the Aggregation and POI layers are filtered by drawing a rectangle.

Applying Geofilters when there is a POI layer (with the apply filters option set to true). Notice that Aggregation and POI layers are filtered

Clicking polygon geo filters:

When polygons are added to the map (either from POI, Stored Layer or WFS sources), clicking on them will create a geo filter.

A geo filter is created when the USA polygon is clicked from the World countries layer

Multiple geo filters:

It is possible to add multiple geo filters to the same Map. When a second geo filter is added, a modal will appear where you can select how you want the second filter to be added.

Even if there are multiple geo filters already on your dashboard. It is possible to overwrite or combine with a specific geo filter by having just that one enabled. Then by creating a new geo filter, the modal will appear and the option selected will apply to the one enabled geo filter.

Modal for Multiple Geofilters

Below, the Aggregation layer and companies POI gets filtered with the applied Geo-filter, while all Populated places remain. This is because Populated places is a Stored Layer and its purpose is to provide context for the Aggregation and POI layers.

Aggregation and POI layer with polygon filter while Store layers remain for reference

Conclusion:

Geospatial analysis is critical for many kinds of investigation. If you want to explore the Siren dashboard with the map yourself, feel free to download the Siren Platform Demo Data and explore the Enhanced Coordinate Map in Siren. 

Resources:

Enhanced Tilemap Configuration

Leaflet Map

Loading Stored Layers

Marker Clustering

GeoJSON

Written by: Manu Agarwal and Edwin Corrigan

OTHER AREAS

Explore our topics

Close