Data Lake
Use Data Lake to archive data received from Telovix eBPF sensors to your own storage. Supported destinations are S3-compatible storage and Azure Blob Storage.
Automatic archives include received sensor payloads, including events, inventory reports, and captured API evidence. An archive cannot contain traffic that was not captured or data that never reached the Console.
Configuring destinations requires Admin access.
Before you start
- Create a bucket or container and grant the Console permission to write to it.
- Make sure the Console can reach the storage endpoint.
- Decide who can access the archived data. Captured content may contain sensitive information.
- Configure retention in the storage service. Console retention does not remove objects from your destination.
Add a destination
- Open Settings > Configuration > Data Lake. Configuration is under Infrastructure on self-hosted deployments or Services on managed deployments.
- Select Configure, or Edit if a destination already exists.
- Select Add destination.
- Choose S3-compatible or Azure Blob Storage, then Continue.
- Enter the destination details.
- Use Test connection, then select Create destination.
S3-compatible storage
| Field | What to enter |
|---|---|
| Destination name | A recognizable name for the archive. |
| Bucket name | An existing bucket. |
| Endpoint URL | Your storage endpoint. Leave blank for AWS S3. |
| Access Key ID / Secret Access Key | The credentials with write access. Omit both only when server-side identity authentication is configured. |
| Region | The destination region. |
| Path prefix | The folder prefix for Telovix objects. The default is telovix-events/. |
Azure Blob Storage
| Field | What to enter |
|---|---|
| Destination name | A recognizable name for the archive. |
| Storage account name | Your Azure storage account. |
| Container name | An existing container. |
| Account Key | The account's Base64 key, not a connection string or SAS token. |
| Path prefix | The folder prefix for Telovix objects. |
| Custom endpoint URL | An alternative endpoint when required by your environment. |
Export settings
Choose an Export interval and Rows per file. The defaults are five minutes and 50,000 rows. These settings control delivery batches, not the sensor's capture frequency.
Leave Enable this destination immediately after saving selected to begin archiving new payloads. Enabling a destination does not backfill older history.
Verify real delivery
A successful connection test confirms that the Console can upload a small test object. It does not confirm that normal archives have already arrived.
- Select Refresh data lake status after sensors report new data.
- Review delivered-payload counts and Last raw upload.
- Open the object identified by Last object in your storage service.
- Confirm it is in the intended bucket or container and prefix.
Automatic archives are Parquet files under <prefix>/raw/..., preserving the received payload content as JSON. Use a Parquet-capable tool to inspect them.
A cleanup warning after a successful test can mean the credentials allow upload but not deletion. Check the message before changing permissions.
Edit, pause, or remove a destination
When editing, leave a credential field blank to retain its saved value. Do not paste a masked credential hint back into the form.
Save changed connection details before testing if you are retaining saved credentials.
If storage is temporarily unreachable, review delivery errors and pending work, restore access, and confirm the backlog is delivered. A prolonged outage can delay ingestion when pending archive capacity is exhausted.
Disabling a destination stops archiving new payloads for it. Re-enabling does not backfill the disabled period. A destination with outstanding deliveries cannot be deleted until that work is resolved.
Export retained event history
Pipeline Health > Export to Parquet sends the last hour of retained runtime-event history to enabled destinations. It does not download a file to your browser and does not reconstruct older inventory reports or uncaptured bodies.
This action requires available Analytics history and fleet-wide Operator or Admin access.
AWS Glue catalog auto-registration is optional and separate from raw archives. It applies to historical runtime-event exports. A Glue status is not confirmation that raw sensor payloads were delivered.