diff --git a/products/logs/src/content/logpush/index.md b/products/logs/src/content/logpush/index.md index a0a77a8e9f9..f8f47370a35 100644 --- a/products/logs/src/content/logpush/index.md +++ b/products/logs/src/content/logpush/index.md @@ -5,21 +5,21 @@ order: 40 # Logpush Service -Have logs of your HTTP requests, Spectrum events, or Firewall events pushed to your cloud service in batches as soon as possible. +Push logs of your HTTP requests, Spectrum events, or Firewall events to your cloud service in batches as soon as possible. Logpush is available to customers on Cloudflare's Enterprise plan. -Logpush can be configured using the [Cloudflare UI (Dashboard)](/logpush/logpush-dashboard/) or [API](/logpush/logpush-configuration-api/). +Configure Logpush using the [Cloudflare UI (Dashboard)](/logpush/logpush-dashboard/) or [API](/logpush/logpush-configuration-api/). If enabling via the API, you can push to the following storage services: - [AWS S3](/logpush/aws-s3/) - [S3-compatible endpoints](/logpush/s3-compatible-endpoints) - [Google Cloud Storage](/logpush/google-cloud-storage/) -- [Microsoft Azure Blobg Storage](/logpush/azure/) +- [Microsoft Azure Blob Storage](/logpush/azure/) -You can also push to the following SIEMs and log management providers: +You also can push to the following SIEMs and log management providers: - [Datadog](/logpush/datadog) - [Splunk](/logpush/splunk) - [Sumo Logic](/logpush/sumo-logic/) -Cloudflare aims to support additional plans and services in the future. Interested in a particular service? Please take this [survey](https://goo.gl/forms/0KpMfae63WMPjBmD2). +Cloudflare aims to support additional plans and services in the future. Interested in a particular service? Take this [survey](https://goo.gl/forms/0KpMfae63WMPjBmD2). diff --git a/products/logs/src/content/logpush/logpush-configuration-api/index.md b/products/logs/src/content/logpush/logpush-configuration-api/index.md index 3ea524bcb67..28a566ffc5e 100644 --- a/products/logs/src/content/logpush/logpush-configuration-api/index.md +++ b/products/logs/src/content/logpush/logpush-configuration-api/index.md @@ -14,8 +14,10 @@ To get started: 1. Set up a storage provider and grant Cloudflare access. For specific instructions: - [Enable Amazon S3](/logpush/aws-s3/) + - [Enable Datadog](/logpush/datadog/) - [Enable Google Cloud Storage](/logpush/google-cloud-storage/) - [Enable Microsoft Azure](/logpush/azure/) + - [Enable Splunk](/logpush/splunk/) - [Enable Sumo Logic](/logpush/sumo-logic/) - [Enable S3-compatible endpoints](/logpush/s3-compatible-endpoints/) diff --git a/products/logs/src/content/logpush/logpush-configuration-api/understanding-logpush-api.md b/products/logs/src/content/logpush/logpush-configuration-api/understanding-logpush-api.md index 5d93c5e8987..e096233bee1 100644 --- a/products/logs/src/content/logpush/logpush-configuration-api/understanding-logpush-api.md +++ b/products/logs/src/content/logpush/logpush-configuration-api/understanding-logpush-api.md @@ -79,9 +79,11 @@ When using Sumo Logic, you may find it helpful to have [Live Tail](https://help. You can specify your cloud service provider destination via the required `destination_conf` parameter. * **AWS S3**: bucket + optional directory + region + optional encryption parameter (if required by your policy); for example: `s3://bucket/[dir]?region=[&sse=AES256]` +* **Datadog**: Datadog endpoint URL + Datadog API key + optional parameters; for example: `datadog://?header_DD-API_KEY=&service=&host=&ddsource=` * **Google Cloud Storage**: bucket + optional directory; for example: `gs://bucket/[dir]` -* **Microsoft Azure**: service-level SAS URL with `https` replaced by `azure` + optional directory added before query string; for example: `azure://[BlobContainerPath]/[dir]?[QueryString]` -* **Sumo Logic**: HTTP source address URL with `https` replaced by `sumo`; for example: `sumo://[SumoEndpoint]/receiver/v1/http/[UniqueHTTPCollectorCode]` +* **Microsoft Azure**: service-level SAS URL with `https` replaced by `azure` + optional directory added before query string; for example: `azure:///[dir]?` +* **Splunk**: Splunk endpoint URL + Splunk channel ID + insecure-skip-verify flag + Splunk sourcetype + Splunk authorization token; for example: `splunk://?channel=&insecure-skip-verify=&sourcetype=&header_Authorization=` +* **Sumo Logic**: HTTP source address URL with `https` replaced by `sumo`; for example: `sumo:///receiver/v1/http/` For S3, Google Cloud Storage, and Azure, logs can be separated into daily subdirectories by using the special string `{DATE}` in the URL path; for example: `s3://mybucket/logs/{DATE}?region=us-east-1&sse=AES256` or `azure://myblobcontainer/logs/{DATE}?[QueryString]`. It will be substituted with the date in `YYYYMMDD` format, like `20180523`. @@ -124,7 +126,7 @@ See a detailed description of the [Logpush object JSON schema](/logpush/logpush- Logpush repeatedly pulls logs on your behalf and uploads them to your destination. -Log options, such fields or sampling rate, are configured in the `logpull_options` job parameter (*see [Logpush job object schema](/logpush/logpush-configuration-api/job-json-schema/)*). If you're migrating from the Logpull API, `logpull_options` is simply the query string for the API call. For example, the following query gets data from the Logpull API: +Log options, such as fields or sampling rate, are configured in the `logpull_options` job parameter (*see [Logpush job object schema](/logpush/logpush-configuration-api/job-json-schema/)*). For example, the following query gets data from the Logpull API: ```bash curl -sv \ @@ -139,9 +141,9 @@ If you don't change any options, you will receive logs with default fields that The three options that you can customize are: -1. Fields; *see [Log fields](/log-fields/)* for the currently available fields. The list of fields is also accessible directly from the API: `https://api.cloudflare.com/client/v4/zones//logpush/datasets//fields`. Default fields: `https://api.cloudflare.com/client/v4/zones//logpush/datasets//fields/default`. -2. Sampling rate; value can range from 0.001 to 1.0 (inclusive). `sample=0.1` means return 10% (1 in 10) of all records. -3. Timestamp format; the format in which timestamp fields will be returned. Value options: unixnano (default), unix, rfc3339. +1. Fields: See *[Log fields](/log-fields/)* for the currently available fields. The list of fields is also accessible directly from the API: `https://api.cloudflare.com/client/v4/zones//logpush/datasets//fields`. Default fields: `https://api.cloudflare.com/client/v4/zones//logpush/datasets//fields/default`. +2. Sampling rate: Value can range from 0.001 to 1.0 (inclusive). `sample=0.1` means return 10% (1 in 10) of all records. +3. Timestamp format: The format in which timestamp fields will be returned. Value options: unixnano (default), unix, rfc3339. To check if `logpull_options` is valid: diff --git a/products/logs/src/content/logpush/logpush-dashboard/index.md b/products/logs/src/content/logpush/logpush-dashboard/index.md index 485b096487d..8d25206a8c0 100644 --- a/products/logs/src/content/logpush/logpush-dashboard/index.md +++ b/products/logs/src/content/logpush/logpush-dashboard/index.md @@ -32,38 +32,35 @@ To enable the Clouflare Logpush service: 6. Under **Select service**, pick your cloud service provider and click **Next** - - -7. For **Amazon S3**, enter or select the following: - * **Bucket path** - * **Daily subfolders** - * **Bucket region** - **Encryption constraint in bucket policy** - * For **Grant Cloudflare access to upload files to your bucket**, make sure your bucket has a policy (if you did not add it already): +7. Complete the steps below based on your provider, then continue to *step 8* + + * For **Amazon S3**, enter or select the following: + * **Bucket path** + * **Daily subfolders** + * **Bucket region** + * **Encryption constraint in bucket policy** + * For **Grant Cloudflare access to upload files to your bucket**, make sure your bucket has a policy (if you did not add it already): * Copy the JSON policy, then go to your bucket in the Amazon S3 console and paste the policy in **Permissions** > **Bucket Policy** and click **Save** -8. For **Google Cloud Storage**, enter or select the following: - * **Bucket path** - * **Daily subfolders** - * For **Grant Cloudflare access to upload files to your bucket**, make sure your bucket has a Cloudflare IAM user (if you did not add it already): - * See *step 2* in [Enable Google Cloud Storage](/logpush/google-cloud-storage/) + * For **Google Cloud Storage**, enter or select the following: + * **Bucket path** + * **Daily subfolders** + * For **Grant Cloudflare access to upload files to your bucket**, make sure your bucket has a Cloudflare IAM user (if you did not add it already): + * See *step 2* in [Enable Google Cloud Storage](/logpush/google-cloud-storage/) -9. For **Microsoft Azure**, enter or select the following: - * **SAS URL** - * See [Enable Microsoft Azure](/logpush/azure/) for instructions on creating this URL - * **Blob container subpath (optional)** - * **Daily subfolders** + * For **Microsoft Azure**, enter or select the following: + * **SAS URL** + * See [Enable Microsoft Azure](/logpush/azure/) for instructions on creating this URL + * **Blob container subpath (optional)** + * **Daily subfolders** -10. For **Sumo Logic**, enter or select the following: - * **HTTP Source Address** - * See [Enable Sumo Logic](/logpush/sumo-logic/) for instructions on getting this address + * For **Sumo Logic**, enter or select the following: + * **HTTP Source Address** + * See [Enable Sumo Logic](/logpush/sumo-logic/) for instructions on getting this address -11. In the Cloudflare dashboard, click **Validate access** +8. In the Cloudflare dashboard, click **Validate access** -12. Follow the on-screen instructions to enter the **Ownership token** (included in a file or log Cloudflare sends to your provider), then click **Prove ownership** +9. Follow the on-screen instructions to enter the **Ownership token** (included in a file or log Cloudflare sends to your provider), then click **Prove ownership** -13. Next in the **Customize log** screen, select the data set in which you're interested (currently `HTTP requests` or `Spectrum events`). You can keep the default fields to include in your log or make changes. You can add or remove fields at a later time by modifying your settings in **Logs** > **Logpush** (select the wrench icon). +10. Next in the **Customize log** screen, select the data set in which you're interested (currently `HTTP requests` or `Spectrum events`). You can keep the default fields to include in your log or make changes. You can add or remove fields at a later time by modifying your settings in **Logs** > **Logpush** (select the wrench icon). -14. To finish enabling Logpush, click **Save and Start Pushing** +11. To finish enabling Logpush, click **Save and Start Pushing** Once connected, Cloudflare lists the provider you just configured under **Logs** > **Logpush**. This is where you can make changes or remove the provider. \ No newline at end of file