Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .source/browser.ts

Large diffs are not rendered by default.

418 changes: 218 additions & 200 deletions .source/server.ts

Large diffs are not rendered by default.

12 changes: 0 additions & 12 deletions content/docs/cloud-provider/_category_.json

This file was deleted.

12 changes: 0 additions & 12 deletions content/docs/cloud-provider/aws/_category_.json

This file was deleted.

12 changes: 0 additions & 12 deletions content/docs/cloud-provider/azure/_category_.json

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ title: "Azure API Service"
---

## API Management Service Logs

Azure API Management (APIM) is a hybrid, multi cloud management platform for APIs across all environments. It provides a scalable, multi-cloud API management platform for securing, publishing, and analyzing APIs.

Azure API Management helps customers:
Expand All @@ -18,11 +19,13 @@ Azure API Management helps customers:
Azure API Management is made up of an API gateway, a management plane, and a developer portal. These components are Azure-hosted and fully managed by default.

## Overview

You can send Azure API Management Service logs to Parseable via Azure Event Hub & Azure Logic Apps. Once logs are unified in Parseable, you can easily monitor and analyze the logs in real-time, and get insights into the performance and usage of your APIs.

Most importantly, you have full ownership and control over this data, so you can use it to improve your API management and security, without cost or privacy concerns.

## Pre-requisites

Before you begin, you will need the following:

- Parseable deployed and running in your environment. Refer the [installation guide](/docs/self-hosted/installation/) for more information.
Expand Down
148 changes: 148 additions & 0 deletions content/docs/ingest-data/cloud/fly-io.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---
title: Fly.io
description: Ingest Fly.io platform logs and metrics into Parseable
---

Ship logs and platform metrics from your [Fly.io](https://fly.io) apps to Parseable. Both pipelines are dedicated Fly apps deployed alongside (not inside) your workload apps; your existing `fly.toml` files are not modified.

## Overview

| Pipeline | Source | Shipper | Parseable route | Availability |
| -------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ----------------------------- | ----------------------------- |
| Logs | Fly's NATS log stream (`nats://[fdaa::3]:4223`) | [`fly-log-shipper`](https://github.com/superfly/fly-log-shipper) with Parseable Vector sink | `POST /api/v1/logstream/...` | All Parseable editions |
| Metrics | Fly's managed Prometheus (`api.fly.io/prometheus/<org>/federate`) | [`parseable-fly-metrics`](https://github.com/parseablehq/parseable-fly-io-metrics) (Go agent) | `POST /v1/prometheus/write` | **Parseable Enterprise / Prism only** |

One agent per Fly org covers every app in that org - no per-app config.

---

## Logs

### Architecture

```
your-app-1 ─┐
your-app-2 ─┤── Fly NATS logs stream ──▶ fly-log-shipper ──▶ Parseable
your-app-N ─┘ (nats://[fdaa::3]:4223) (Fly app) /api/v1/logstream/<stream>
```

### Deploy

`fly-log-shipper` is Fly's official log forwarder, with a Parseable sink available in the upstream repo (see [PR #76](https://github.com/superfly/fly-log-shipper/pull/76)).

```bash
git clone https://github.com/superfly/fly-log-shipper
cd fly-log-shipper

# create the shipper app in your Fly org
flyctl apps create <org>-fly-log-shipper --org <your-fly-org>

# auth + Parseable sink secrets
flyctl secrets set \
ACCESS_TOKEN=$(flyctl auth token) \
ORG=<your-fly-org-slug> \
PARSEABLE_URL=https://<your-parseable-host> \
PARSEABLE_USERNAME=<user> \
PARSEABLE_PASSWORD=<pass> \
PARSEABLE_STREAM=fly_logs

flyctl deploy --ha=false
```

Logs from every app in the org now flow into the `fly_logs` stream. Refer to the [fly-log-shipper README](https://github.com/superfly/fly-log-shipper#sinks) for the full list of sink env vars.

---

## Metrics

<OfferingPills pro enterprise className="mb-4" />

The Fly metrics pipeline writes via Parseable's `POST /v1/prometheus/write` route, which is available in **Parseable Enterprise** and **Prism** (distributed) deployments. The open-source Parseable server does not expose this route.

### Architecture

```
your-app-1 ─┐
your-app-2 ─┤── Fly managed Prometheus ──▶ parseable-fly-metrics ──▶ Parseable
your-app-N ─┘ api.fly.io/prometheus/<org>/federate (Fly app) /v1/prometheus/write
(ingestor node)
```

### Deploy

```bash
git clone https://github.com/parseablehq/parseable-fly-metrics
cd parseable-fly-metrics

# edit fly.toml - set `app = "<your-unique-name>"` and `primary_region`

flyctl apps create <your-unique-name> --org <your-fly-org>

flyctl secrets set \
FLY_ORG=<your-fly-org-slug> \
FLY_METRICS_TOKEN=$(flyctl auth token) \
PARSEABLE_URL=https://<your-parseable-ingestor>:8010 \
PARSEABLE_USERNAME=<user> \
PARSEABLE_PASSWORD=<pass>

flyctl deploy --ha=false
```

Expected log line per scrape interval:

```
scrape ok: families=61 series=134 samples=134 duration=460ms
```

`PARSEABLE_URL` must point at a Prism **ingestor** node, not the query node. Run a single agent machine per org; multiple instances would double-scrape.

### Configuration

| Variable | Required | Default | Notes |
| ---------------------- | -------- | ------------- | -------------------------------------------------------------------------------------------------- |
| `FLY_ORG` | yes | - | Fly org slug. Find via `flyctl orgs list`. |
| `FLY_METRICS_TOKEN` | yes | - | Output of `flyctl auth token` or `flyctl tokens create readonly -o <org>`. |
| `PARSEABLE_URL` | yes | - | Base URL of the Parseable ingestor. |
| `PARSEABLE_USERNAME` | yes | - | HTTP basic auth username. |
| `PARSEABLE_PASSWORD` | yes | - | HTTP basic auth password. |
| `PARSEABLE_STREAM` | no | `fly_metrics` | Target stream name. Parseable creates the stream automatically on first write. |
| `SCRAPE_INTERVAL_SECS` | no | `30` | Seconds between scrapes. |

---

## Verify

Query the Parseable query node:

```bash
curl -u <user>:<pass> -X POST \
-H 'Content-Type: application/json' \
-d '{"query":"select metric_name, app, region, data_point_value from fly_metrics order by p_timestamp desc limit 5","startTime":"1h","endTime":"now"}' \
https://<your-parseable-query-node>/api/v1/query
```

Sample row from the `fly_metrics` stream:

```json
{
"app": "hellofly",
"region": "sin",
"instance": "9080d161a52928",
"metric_name": "fly_instance_memory_buffers",
"metric_type": "gauge",
"data_point_value": 778240,
"time_unix_nano": "2026-05-18T06:45:38.637",
"scope_name": "prometheus_remote_write",
"p_format": "otel-metrics"
}
```

For logs, query the `fly_logs` stream the same way.

---

## Notes

- **Auth scheme.** Fly's Prometheus endpoint requires `Authorization: FlyV1 <token>` for tokens issued by recent `flyctl` versions. The agent handles this automatically; the `Bearer` scheme returns HTTP 401.
- **Idle apps.** Apps with `auto_stop_machines = "stop"` stop emitting metrics until they are woken. Fly's `/federate` returns HTTP 200 with an empty body in that state.
- **Self-observability blind spot.** Both agents (logs and metrics) are themselves Fly apps in the same org, so they appear in their own data. If an agent crashes, its own outage is invisible from Parseable; set a stream-silence alert as a watchdog.
7 changes: 6 additions & 1 deletion content/docs/ingest-data/cloud/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,13 @@
"title": "Cloud Providers",
"pages": [
"aws-cloudwatch",
"aws-lambda",
"aws-data-firehose",
"aws-kinesis",
"azure-event-hubs",
"gcp-pubsub"
"azure-api-service",
"azure-service-bus",
"gcp-pubsub",
"fly-io"
]
}
10 changes: 10 additions & 0 deletions content/docs/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,16 @@ Observability pipeline

---

## Application platforms

<Cards>
<Card href="/docs/integrations/platforms/fly-io" title="Fly.io">
Ingest Fly.io platform logs and metrics
</Card>
</Cards>

---

## Container orchestration

<Cards>
Expand Down
1 change: 1 addition & 0 deletions content/docs/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
"api",
"---Self Hosted---",
"self-hosted/installation",
"self-hosted/aws",
"self-hosted/configuration",
"self-hosted/storage-targets",
"self-hosted/metrics",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
---
title: "AWS"
---
Amazon Web Services (AWS) is the leading cloud services provider, offering a wide range of services including compute, storage, databases, analytics, machine learning, and more. Parseable integrates with AWS to provide a seamless experience for our customers.

This document provides an overview of the AWS services that Parseable integrates with, and how to set up the integration.

### Authentication and access control

Parseable uses AWS IAM roles to authenticate and authorize access to AWS services. You can create an IAM role with the necessary permissions and attach it to the Parseable instance to enable access to AWS services.

### Deployment and configuration

Parseable can be deployed in AWS using the following methods:

- AWS EKS: Install Parseable in your EKS cluster using [this guide here](/docs/self-hosted/installation/standalone/aws-eks).
Expand Down
1 change: 1 addition & 0 deletions content/docs/self-hosted/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"defaultOpen": true,
"pages": [
"installation",
"aws",
"configuration",
"storage-targets",
"metrics",
Expand Down