Whitepaper

BYODb Architecture Deep Dive

How Bring Your Own Database works and why it changes everything

Introduction

Traditional SIEMs require you to ship all your security data to their infrastructure. This creates data lock-in, escalating costs, and compliance headaches. Vigilense AI's BYODb (Bring Your Own Database) architecture flips this model your data stays where it is, and we bring the intelligence to you.

The Problem with Traditional SIEM Data Architecture

Cost Explosion

Volume-based pricing means costs grow exponentially as your environment scales. Organizations often reduce visibility to control costs.

Vendor Lock-in

Once your data is in their format and infrastructure, migration becomes a multi-year project. You're trapped.

Data Sovereignty

Shipping data across borders creates compliance issues for GDPR, data residency requirements, and industry regulations.

Latency

Moving petabytes of data introduces delay. Real-time detection requires data to travel to and from vendor infrastructure.

How BYODb Works

Architecture Overview

BYODb is built on a simple principle: move compute to data, not data to compute.

  1. Connection: You provide secure, read-only access to your existing data stores
  2. Schema Discovery: Our system automatically maps your data schema and normalizes field names
  3. Query Federation: Queries are translated to your database's native format and executed locally
  4. Detection Storage: Detection results and alert metadata are stored securely to enable investigation and response
  5. Intelligence Delivery: Alerts, insights, and recommendations are delivered to you

Supported Databases

Search & Analytics

  • OpenSearch / Elasticsearch
  • Security Lakes
  • Sumo Logic
  • Datadog

Cloud Data Warehouses

  • Snowflake
  • BigQuery
  • Databricks
  • Amazon Redshift

Object Storage

  • AWS S3
  • Azure Blob Storage
  • Google Cloud Storage
  • MinIO

Databases

  • PostgreSQL
  • MongoDB
  • ClickHouse
  • TimescaleDB

Security Model

Credential Management

  • Encryption: All credentials encrypted at rest using HSM-backed keys
  • Secrets Manager Integration: Optional integration with HashiCorp Vault, AWS Secrets Manager, Azure Key Vault
  • Rotation: Support for automatic credential rotation
  • Audit: All credential access is logged

Access Controls

  • Read-Only: We only require read access no write operations
  • Least Privilege: Access scoped to specific tables/indices needed
  • IP Whitelisting: Connections originate from known IP ranges
  • Private Connectivity: VPC peering, PrivateLink, and VPN options available

Data Handling

  • Minimal Storage: Only detection results and alert metadata are stored securely
  • Raw Data Stays In Place: Source logs are never copied or moved to Vigilense infrastructure
  • No Export: Data cannot be bulk exported or downloaded
  • Audit Trail: All queries logged for compliance

Performance Optimization

Query Optimization

Our query engine is optimized for each database type:

  • Native query translation (SQL, DSL, etc.)
  • Query plan optimization
  • Parallel query execution
  • Result streaming for large datasets
  • Intelligent caching of metadata (not data)

Performance Benchmarks

Database Avg Query Time Data Volume
OpenSearch < 500ms 10TB+
Snowflake < 2s 100TB+
S3 + Athena < 5s 1PB+

Implementation Guide

Step 1: Inventory Your Data

Document your current security data sources and where they're stored. Common sources include:

  • Firewall and network logs
  • Endpoint detection data
  • Cloud audit trails (CloudTrail, Azure Activity Log)
  • Identity and access logs
  • Application security logs

Step 2: Create Service Account

Create a dedicated service account with read-only access to your security data. We provide database-specific guides for each supported platform.

Step 3: Configure Connection

Provide connection details through our secure onboarding wizard. Credentials are encrypted before transmission and stored in our HSM-backed vault.

Step 4: Schema Mapping

Our system automatically discovers your schema. You can customize field mappings to align with your naming conventions.

Step 5: Validate & Go Live

Run validation queries to ensure connectivity and permissions. Once validated, enable real-time monitoring.

Case Study: Financial Services

A Fortune 500 financial services company implemented BYODb to address strict data residency requirements:

  • Challenge: Could not move security data outside their private cloud due to regulations
  • Solution: BYODb connected to their existing Snowflake deployment
  • Result: Full SIEM capabilities without any data leaving their environment
  • Savings: $2.3M annually vs. traditional SIEM data ingestion costs

Get Started

Ready to keep your data where it belongs? Contact us for a technical deep dive on implementing BYODb in your environment.