We’re super excited to announce today Siren 10.4. This latest version of Siren introduces several major features among which:
- Efficient (cluster distributed) Shortest Path capabilities.
- OpenID support
- Support for Internationalization and White-labeling
- IBM i2 Analyst Notebook compatibility
- Export configuration object trees
- New documentation format
Let’s start with one which represents a breakthrough in what one can do on data stored in Elasticsearch.
Efficient graph “Shortest path” finding capabilities
Finding the shortest and most significant path across connected records is critical in sectors such as intelligence, law enforcement, and financial fraud. This version of Siren introduces an implementation that works natively on the backends that Siren is connected to be it SQL or Elasticsearch – in the latter case making use of our Federate plug-in for distributed, parallelized execution.
This version replaces Siren’s simpler implementation which operated within the constraints of memory of a single JVM (and required fetching a lot of data).
The new functionality can be called from the Link Analysis component. In the following video, the shortest path between two people is found in a 100M+ index. of Call Data Records (CDRs):
A quick look under the hood
Under the hood, this algorithm works in parallelized fashion leveraging the scalability of the Federate Semi-joins operations.
First, it analyzes the data model, listing all the possible “ontological paths” between the 2 nodes. Then it builds a “query trees” which spawns from the original nodes and has a condition that the trees must match on some unknown, yet existing, node.
This node will be the “center” of the shortest path that we’re trying to find. While this sounds like an exponentially complex operation to process (due to the path length), in practice, the complexity expands linearly most of the time thanks to the way Federate operates. We’ll describe this implementation, now covered by 3 pending patents, in a future blog post. For now, give it a try on your favorite connected data and let us know how it goes!
Further improvements to graph database support
While implementing some native Graph algorithms directly in Elasticsearch – and other SQL backends that Siren supports – can be quite useful for some operational use cases, Siren is also committed to supporting more and more native Graph databases.
We’re happy to say that Siren 10.4 further improves support for Neo4j and that we’re working for more graph database compatibility in future versions.
OpenID Support
Siren now natively supports OpenID authentication and authorization, generating automatically a valid token (from client certificate) without passing through external Identity and Access Management software.
Internationalization (I18N) and White-labeling
With the localization support, Siren now makes it easy for you to translate the Siren application content in your local language and allow non-English speaking users to efficiently use the platform.
We currently support 7 locales – Spanish, French German, Japanese, Korean, Chinese and Arabic. The localization support also extends to any custom plugins that you’ve set up for your Siren platform.
Also, partners creating solutions with Siren now have an easy way to White-label Siren into their product by changing names, logos, and icons. Together with a custom CSS, this allows Siren to better fit it into larger integrations.
Export to IBM i2 Analyst Notebook (Siren graphs -> .ANB)
IBM i2 Analyst Notebook is by far the world most deployed tool for Link Analysis in Law Enforcement and Intelligence worldwide. Siren and i2 are pretty complementary, with Siren allowing “Wide to Narrow” big data drill-downs (and then later big data monitoring) and Analyst Notebook providing sophisticated refinements and well-known Iconography and ability to generate a “picture perfect” graph which is then used in court.
With Siren 10.4 and as part of our IBM Business Partnership we’re very excited to announce that Siren generated Graphs can now be opened directly by i2 Analyst Notebook, making Siren-i2 workflows a breeze.
The feature is quite streamlined. A UI assisted editor allows administrators to map datatypes from Siren into i2 types. The graph then allows exports in ANB format which can be directly opened in graph carrying over metadata/types, timestamps and geographical locations, graph layout and more. A quick demo:
Export configuration object trees
Exporting and importing configuration objects is important when managing multiple Siren instances. Objects, however, are interlinked (E.g. dashboards → visualizations → searches → index pattern definitions).
In Siren 10.4 a dependency tree is calculated and one gets the option to export it completely and then easily import it into another instance.
Drag and Drop POI layers on the Map
You can now drag and drop a selection of geo-annotated records or Point of Interest (POI) layers on the map – just as you do on the Graph.
This is an analyst driven operation (not a configuration setting), for greater flexibility.
Others
- Added a Start.bat script in the Windows releases, making it easier to start the platform for Windows desktop users.
- Topic Clustering component is considerably faster on larger Datasets
- Added the ability to refresh counts on the graph (update button). Counts are also automatically and instantaneously refreshed when deselecting relations from the “expansion” panel.
- Support for more advanced Graph Lens types.
What’s next? A sneak peek at Siren 10.5
Siren 10.5 is well on its way with a likely May release. We’re excited about what’s coming next so here is a preview:
- Elasticsearch 7.x compatibility (note Federate for Elasticsearch 7 is already available today)
- Graph “Side Panels” – see details and metadata of the selected nodes at the side of the graph. Fully customizable/scriptable allowing to serve multiple use cases.
- Siren scripting frameworks – create custom workflows in Siren (when this do that) as well as custom “panels” with buttons, data entry spaces and others which can automate navigation and perform custom actions
- (New) Siren Web Service integration framework: call web services from dashboards and from the graph. Automatically index responses for later reuse, forming a “knowledge graph” on demand.
- Siren NLP – Built-in plugin for Natural Language Processing
We’ll be writing more about these shortly, so stay tuned.
Siren 10.4 is now available for download and as always looking forward to hearing from you on our community.
Also published on Medium.
