The OpenCTI platform technological stack has been designed to be able to scale horizontally. All dependencies such as Elastic or Redis can be deployed in cluster mode and performances can be drastically increased by deploying multiple platform and worker instances.
High level architecture
Here is the high level architecture for customers and Filigran cloud platform to ensure both high availability and throughput.
In the ElasticSearch configuration of OpenCTI, it is possible to declare all nodes.
OpenCTI is also compatible with OpenSearch and AWS / GCP / Azure native search services based on the ElasticSearch query language.
Redis should be turned to cluster mode:
OpenCTI is also compatible with ElastiCache, MemoryStore and AWS / GCP / Azure native services based on the Redis protocol.
For the RabbitMQ cluster, you will need a TCP load balancer on top of the nodes since the configuration does not support multi-nodes for now:
OpenCTI is also compatible with Amazon MQ, CloudAMQP and AWS / GCP / Azure native services based on the AMQP protocol.
S3 bucket / MinIO
MinIO is an open source server able to serve S3 buckets. It can be deployed in cluster mode and is compatible with several storage backend. OpenCTI is compatible with any tool following the S3 standard.
As showed on the schema, best practices for cluster mode and to avoid any congestion in the technological stack are:
- Deploy platform(s) dedicated to end users and connectors registration
- Deploy platform(s) dedicated to workers / ingestion process
- We recommend 3 to 4 workers maxiumum by OpenCTI instance.
- The ingestion platforms will never be accessed directly by end users.
When enabling clustering, the number of nodes is displayed in Settings > Parameters.
Managers and schedulers
Also, since some managers like the rule engine, the task manager and the notification manager can take some resources in the OpenCTI NodeJS process, it is highly recommended to disable them in the frontend cluster. OpenCTI automatically handle the distribution and the launching of the engines across all nodes in the cluster except where they are explicitely disabled in the configuration.