Skip to content

XTM Composer architecture

Overview

XTM Composer is a micro-orchestration tool that manages connectors/collectors/injectors for Filigran platforms (OpenCTI and OpenAEV). Written in Rust for performance and reliability, it acts as a bridge between the platforms and container orchestration systems.

Global mechanism

Core architecture flow

┌─────────────┐     REST/GraphQL     ┌──────────────┐
│   OpenCTI   │◄────────────────────►│              │
│   Platform  │                      │  XTM         │     Container API
└─────────────┘                      │  Composer    │◄─────────────────────►┌─────────────┐
                                     │              │                       │ Orchestrator│
┌─────────────┐                      │              │                       │ (K8s/Docker)│
│   OpenAEV   │◄────────────────────►│              │                       └─────────────┘
│   Platform  │     REST/GraphQL     └──────────────┘
└─────────────┘

Main components

  1. Engine Module (src/engine/)
  2. Manages lifecycle of orchestration and health monitoring
  3. Creates separate async tasks for each platform (OpenCTI/OpenAEV)

  4. Handles graceful shutdown via system signals

  5. API Module (src/api/)

  6. Abstracts platform communication through ComposerApi trait
  7. Handles GraphQL queries/mutations for OpenCTI

  8. Manages container configuration retrieval and status updates

  9. Orchestrator Module (src/orchestrator/)

  10. Implements Orchestrator trait for different container systems
  11. Supports Kubernetes, Docker, and Portainer

  12. Handles container lifecycle operations

  13. Config Module (src/config/)

  14. Manages application settings from YAML files
  15. Supports environment-specific configurations
  16. Handles credentials and encryption keys

Terminology

XTM Composer orchestrates different types of containerized components depending on the platform:

  • OpenCTI: Deploys Connectors as containers (import, export, stream)
  • OpenAEV: Deploys Collectors and Injectors as containers

Each component is deployed and managed as a container:

Platform Component Types Deployed As
OpenCTI Connectors Container
OpenAEV Collectors/Injectors Container

This abstraction allows the composer to use the same orchestration logic regardless of the platform-specific terminology.

Execution flow

  1. Initialization
  2. Load configuration from config/default.yaml and environment
  3. Initialize RSA private key for configuration decryption

  4. Set up logging system

  5. Platform Registration

  6. Connect to platform (OpenCTI/OpenAEV)
  7. Register composer instance with unique manager ID

  8. Verify API compatibility

  9. Main Orchestration Loop

  10. Execute every execute_schedule seconds (default: 30s)
  11. Pull connector configurations
  12. Reconcile desired vs actual state
  13. Apply necessary changes

Pull mechanism

Configuration retrieval (Connectors/Collectors/Injectors)

The composer periodically pulls container configurations from the platform:

// Executed every 30 seconds by default
async fn orchestrate() {
    // 1. Fetch all container configurations from platform
    // (connectors for OpenCTI, collectors/injectors for OpenAEV)
    let connectors = api.connectors().await; // Generic method name

    // 2. For each container configuration
    for connector in connectors {
        // 3. Check if container exists in orchestrator
        let container = orchestrator.get(connector).await;

        // 4. Reconcile state
        match container {
            Some(c) => orchestrate_existing(c, connector),
            None => orchestrate_missing(connector)
        }
    }
}

Container configuration structure

Each container configuration (running a connector/collector/injector) contains: - ID: Unique identifier - Name: Human-readable name - Image: Docker image to deploy - Contract Hash: Version identifier for configuration - Status: Current and requested states - Configuration: Key-value pairs (potentially encrypted)

State synchronization

The composer maintains two-way synchronization:

  1. Platform → Composer (Pull)
  2. Fetch latest component definitions (connectors/collectors/injectors)
  3. Retrieve requested status changes

  4. Get configuration updates

  5. Composer → Platform (Push)

  6. Report actual container status
  7. Send container logs
  8. Update health metrics

Update mechanism

Container lifecycle management

The update mechanism handles several scenarios:

1. Deployment (Missing Container)

// Container doesn't exist but component is defined in platform
// (connector for OpenCTI, collector/injector for OpenAEV)
async fn orchestrate_missing(connector) {
    orchestrator.deploy(connector).await;
    api.patch_status(connector.id, ConnectorStatus::Stopped).await;
}

2. Status Alignment

// Align requested status with actual state
match (requested_status, current_status) {
    (RequestedStatus::Starting, ConnectorStatus::Stopped) => {
        orchestrator.start(container).await;
    }
    (RequestedStatus::Stopping, ConnectorStatus::Started) => {
        orchestrator.stop(container).await;
    }
}

3. Configuration Updates

// Check if configuration hash changed
if requested_hash != current_hash {
    orchestrator.refresh(connector).await;  // Recreate with new config
}

4. Health Monitoring

  • Tracks container restart count
  • Detects reboot loops (>3 restarts in <5 minutes)
  • Reports health metrics every 30 seconds for running containers

5. Log Collection

  • Collects container logs every logs_schedule minutes
  • Sends logs to platform for centralized monitoring
  • Maintains log history for debugging

Cleanup process

The composer automatically removes orphaned containers: 

// Remove containers not defined in platform
// (connectors for OpenCTI, collectors/injectors for OpenAEV)
for container in orchestrator.list().await {
    if !platform_connectors.contains(container.id) {
        orchestrator.remove(container).await;
    }
}

Configuration encryption

Hybrid encryption scheme

XTM Composer uses RSA/AES hybrid encryption for sensitive configuration:

Platform                    Composer
────────                    ────────
1. Generate AES key
2. Encrypt config with AES
3. Encrypt AES key with RSA public key
4. Send: [Version|Encrypted AES Key|Encrypted Data]
                            │
                            ▼
                    5. Decrypt AES key with RSA private key
                    6. Decrypt config with AES key
                    7. Use plaintext configuration

Encryption format

Encrypted values are Base64-encoded with structure: 

[1 byte: Version][512 bytes: RSA-encrypted AES key+IV][N bytes: AES-encrypted data]

Key management

  1. RSA Private Key
  2. Loaded at startup from file or environment variable
  3. PKCS#8 PEM format required

  4. Used to decrypt AES keys

  5. AES Encryption

  6. AES-256-GCM for data encryption
  7. Random key and IV per configuration value
  8. Provides authenticated encryption

Security properties

  • Confidentiality: Sensitive data encrypted at rest and in transit
  • Key Isolation: Each configuration value has unique AES key
  • Forward Secrecy: Compromised AES key doesn't affect other values
  • Authentication: GCM mode provides data integrity

Performance considerations

  1. Async/Await: All I/O operations are asynchronous
  2. Periodic Execution: Configurable intervals prevent API flooding
  3. Batch Processing: Multiple connectors processed in single cycle
  4. Resource Efficiency: Rust's zero-cost abstractions minimize overhead
  5. Error Recovery: Graceful handling of failures with retry logic

Fault tolerance

  • Health Checks: Regular ping to platform ensures connectivity
  • Version Detection: Re-registers on platform upgrade
  • Reboot Loop Detection: Prevents infinite restart cycles
  • Graceful Shutdown: Handles system signals properly
  • State Persistence: Platform maintains desired state