[New Package] Azure Frontdoor Integration #2497
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| dependencies: | ||
| ecs: | ||
| reference: git@8.6 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Azure Frontdoor Logs Integration | ||
|
|
||
| The azure frontdoor logs integration retrieves different types of log data from AFD. | ||
| Azure Front Door provides different logging to help you track, monitor, and debug your Front Door. | ||
|
|
||
| - Access logs have detailed information about every request that AFD receives and help you analyze and monitor access patterns, and debug issues. | ||
| - Activity logs provide visibility into the operations done on Azure resources. | ||
| - Health Probe logs provides the logs for every failed probe to your origin. | ||
| - Web Application Firewall (WAF) logs provide detailed information of requests that gets logged through either detection or prevention mode of an Azure Front Door endpoint. A custom domain that gets configured with WAF can also be viewed through these logs. | ||
|
|
||
| ## Data streams | ||
|
|
||
| This integration collects two types of data streams: | ||
|
|
||
| - access log | ||
| - waf logs | ||
|
|
||
| ## Requirements | ||
|
|
||
| ### Credentials | ||
|
|
||
| `eventhub` : | ||
| _string_ | ||
| Is the fully managed, real-time data ingestion service. | ||
|
|
||
| `consumer_group` : | ||
| _string_ | ||
| The publish/subscribe mechanism of Event Hubs is enabled through consumer groups. A consumer group is a view (state, position, or offset) of an entire event hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets. | ||
| Default value: `$Default` | ||
|
|
||
| `connection_string` : | ||
| _string_ | ||
| The connection string required to communicate with Event Hubs, steps here https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-get-connection-string. | ||
|
|
||
| A Blob Storage account is required in order to store/retrieve/update the offset or state of the eventhub messages. This means that after stopping the filebeat azure module it can start back up at the spot that it stopped processing messages. | ||
|
|
||
| `storage_account` : | ||
| _string_ | ||
| The name of the storage account the state/offsets will be stored and updated. | ||
|
|
||
| `storage_account_key` : | ||
| _string_ | ||
| The storage account key, this key will be used to authorize access to data in your storage account. | ||
|
|
||
| `resource_manager_endpoint` : | ||
| _string_ | ||
| Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. | ||
| Ex: | ||
| https://management.chinacloudapi.cn/ for azure ChinaCloud | ||
| https://management.microsoftazure.de/ for azure GermanCloud | ||
| https://management.azure.com/ for azure PublicCloud | ||
| https://management.usgovcloudapi.net/ for azure USGovernmentCloud | ||
| Users can also use this in case of a Hybrid Cloud model, where one may define their own endpoints. | ||
|
|
||
| ## Acceess Logs | ||
|
|
||
| {{fields "access"}} | ||
|
|
||
| ## WAF Logs | ||
|
|
||
| {{fields "waf"}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # newer versions go on top | ||
| - version: "0.0.1" | ||
| changes: | ||
| - description: Initial draft of the package | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/2497 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| dynamic_fields: | ||
| event.ingested: ".*" | ||
| fields: | ||
| tags: | ||
| - preserve_original_event | ||
| - azure-frontdoor-access |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"category":"FrontdoorAccessLog","operationName":"Microsoft.Network/FrontDoor/AccessLog/Write","properties":{"ErrorInfo":"NoError","backendHostname":"samplev6erp.azurewebsites.net:443","cacheStatus":"CONFIG_NOCACHE","clientIp":"89.160.20.128","clientPort":"50382","httpMethod":"POST","httpStatusCode":"200","httpStatusDetails":"200","httpVersion":"2.0.0.0","isReceivedFromClient":true,"pop":"SIN","requestBytes":"2545","requestProtocol":"HTTPS","requestUri":"https://erp.testcloud.com:443/StockSetup/GetStockListByCir","responseBytes":"1205","routingRuleName":"erp","rulesEngineMatchNames":[],"securityProtocol":"TLS 1.2","socketIp":"89.160.20.128","timeTaken":"0.384","timeToFirstByte":"0.384","trackingReference":"0k1y5YQAAAAAWd0Uc6UcnR7WN8uo2prYZU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=","userAgent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36"},"resourceId":"/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD","time":"2021-12-15T03:10:11.6479719Z"} | ||
| {"category":"FrontdoorAccessLog","operationName":"Microsoft.Network/FrontDoor/AccessLog/Write","properties":{"ErrorInfo":"NoError","backendHostname":"samplev6erp.azurewebsites.net:443","cacheStatus":"CONFIG_NOCACHE","clientIp":"175.16.199.0","clientPort":"6610","httpMethod":"GET","httpStatusCode":"200","httpStatusDetails":"200","httpVersion":"2.0.0.0","isReceivedFromClient":true,"pop":"SIN","requestBytes":"1984","requestProtocol":"HTTPS","requestUri":"https://erp.testcloud.com:443/saleInvoice/readBySyskeySIByRoleAllowed/2112140619239361392","responseBytes":"2308","routingRuleName":"erp","rulesEngineMatchNames":[],"securityProtocol":"TLS 1.2","socketIp":"175.16.199.0","timeTaken":"0.122","timeToFirstByte":"0.122","trackingReference":"0lWK5YQAAAAD89Q/jewlnT7dWvZNIh72LU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=","userAgent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36"},"resourceId":"/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD","time":"2021-12-15T03:35:49.9266300Z"} | ||
| {"category":"FrontdoorAccessLog","operationName":"Microsoft.Network/FrontDoor/AccessLog/Write","properties":{"ErrorInfo":"NoError","backendHostname":"samplev6erp.azurewebsites.net:443","cacheStatus":"CONFIG_NOCACHE","clientIp":"175.16.199.0","clientPort":"6610","httpMethod":"GET","httpStatusCode":"200","httpStatusDetails":"200","httpVersion":"2.0.0.0","isReceivedFromClient":true,"pop":"SIN","requestBytes":"1971","requestProtocol":"HTTPS","requestUri":"https://erp.testcloud.com:443/Customer/searchContactList/2107050813256062892","responseBytes":"637","routingRuleName":"erp","rulesEngineMatchNames":[],"securityProtocol":"TLS 1.2","socketIp":"175.16.199.0","timeTaken":"0.064","timeToFirstByte":"0.064","trackingReference":"0lWK5YQAAAAAnKnstK4rwSovl+unjuKhoU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=","userAgent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36"},"resourceId":"/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD","time":"2021-12-15T03:35:50.0584922Z"} | ||
| {''"records"'': [{"time":"2021-02-02T07:15:37.3640748Z","resourceId":"/SUBSCRIPTIONS/saDFEEQW-JESSIE","category":"FrontdoorAccessLog"}]} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,290 @@ | ||
| { | ||
| "expected": [ | ||
| { | ||
| "@timestamp": "2021-12-15T03:10:11.647Z", | ||
| "azure": { | ||
| "frontdoor": { | ||
| "access": { | ||
| "backend_hostname": "samplev6erp.azurewebsites.net:443", | ||
| "cache_status": "CONFIG_NOCACHE", | ||
| "error_info": "NoError", | ||
| "is_received_from_client": true, | ||
| "pop": "SIN", | ||
| "routing_rule_name": "erp", | ||
| "rules_engine_match_names": [], | ||
| "time_taken": "0.384", | ||
| "time_to_first_byte": "0.384" | ||
| }, | ||
| "category": "FrontdoorAccessLog", | ||
| "operation_name": "Microsoft.Network/FrontDoor/AccessLog/Write", | ||
|
Contributor
There was a problem hiding this comment. "operation_name" seems like an action is taking place, this can be made more verbose by treating it so. Example we can add it as part of the event object |
||
| "resource_id": "/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD", | ||
| "tracking_reference": "0k1y5YQAAAAAWd0Uc6UcnR7WN8uo2prYZU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=" | ||
| } | ||
| }, | ||
| "client": { | ||
| "address": "89.160.20.128", | ||
| "ip": "89.160.20.128", | ||
| "port": 50382 | ||
| }, | ||
| "cloud": { | ||
| "provider": "azure" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.6.0" | ||
| }, | ||
| "event": { | ||
| "category": [ | ||
| "network" | ||
| ], | ||
| "type": [ | ||
| "connection" | ||
| ] | ||
| }, | ||
| "http": { | ||
| "request": { | ||
| "bytes": 2545, | ||
| "method": "POST" | ||
| }, | ||
| "response": { | ||
| "bytes": 1205, | ||
| "status_code": 200 | ||
| }, | ||
| "version": "2.0.0.0" | ||
| }, | ||
| "network": { | ||
| "protocol": "HTTPS" | ||
| }, | ||
| "source": { | ||
| "as": { | ||
| "number": 29518, | ||
| "organization": { | ||
| "name": "Bredband2 AB" | ||
| } | ||
| }, | ||
| "geo": { | ||
| "city_name": "Linköping", | ||
| "continent_name": "Europe", | ||
| "country_iso_code": "SE", | ||
| "country_name": "Sweden", | ||
| "location": { | ||
| "lat": 58.4167, | ||
| "lon": 15.6167 | ||
| }, | ||
| "region_iso_code": "SE-E", | ||
| "region_name": "Östergötland County" | ||
| } | ||
| }, | ||
| "tags": [ | ||
| "preserve_original_event", | ||
| "azure-frontdoor-access" | ||
| ], | ||
| "tls": { | ||
| "version": "1.2", | ||
| "version_protocol": "TLS" | ||
| }, | ||
| "url": { | ||
| "original": "https://erp.testcloud.com:443/StockSetup/GetStockListByCir" | ||
| }, | ||
| "user_agent": { | ||
| "device": { | ||
| "name": "Other" | ||
| }, | ||
| "name": "Chrome", | ||
| "original": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36", | ||
| "os": { | ||
| "full": "Windows 7", | ||
| "name": "Windows", | ||
| "version": "7" | ||
| }, | ||
| "version": "96.0.4664.93" | ||
| } | ||
| }, | ||
| { | ||
| "@timestamp": "2021-12-15T03:35:49.926Z", | ||
| "azure": { | ||
| "frontdoor": { | ||
| "access": { | ||
| "backend_hostname": "samplev6erp.azurewebsites.net:443", | ||
| "cache_status": "CONFIG_NOCACHE", | ||
| "error_info": "NoError", | ||
| "is_received_from_client": true, | ||
| "pop": "SIN", | ||
| "routing_rule_name": "erp", | ||
| "rules_engine_match_names": [], | ||
| "time_taken": "0.122", | ||
| "time_to_first_byte": "0.122" | ||
| }, | ||
| "category": "FrontdoorAccessLog", | ||
| "operation_name": "Microsoft.Network/FrontDoor/AccessLog/Write", | ||
| "resource_id": "/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD", | ||
| "tracking_reference": "0lWK5YQAAAAD89Q/jewlnT7dWvZNIh72LU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=" | ||
| } | ||
| }, | ||
| "client": { | ||
| "address": "175.16.199.0", | ||
| "ip": "175.16.199.0", | ||
| "port": 6610 | ||
| }, | ||
| "cloud": { | ||
| "provider": "azure" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.6.0" | ||
| }, | ||
| "event": { | ||
| "category": [ | ||
| "network" | ||
| ], | ||
| "type": [ | ||
| "connection" | ||
| ] | ||
| }, | ||
| "http": { | ||
| "request": { | ||
| "bytes": 1984, | ||
| "method": "GET" | ||
| }, | ||
| "response": { | ||
| "bytes": 2308, | ||
| "status_code": 200 | ||
| }, | ||
| "version": "2.0.0.0" | ||
| }, | ||
| "network": { | ||
| "protocol": "HTTPS" | ||
| }, | ||
| "source": { | ||
| "geo": { | ||
| "city_name": "Changchun", | ||
| "continent_name": "Asia", | ||
| "country_iso_code": "CN", | ||
| "country_name": "China", | ||
| "location": { | ||
| "lat": 43.88, | ||
| "lon": 125.3228 | ||
| }, | ||
| "region_iso_code": "CN-22", | ||
| "region_name": "Jilin Sheng" | ||
| } | ||
| }, | ||
| "tags": [ | ||
| "preserve_original_event", | ||
| "azure-frontdoor-access" | ||
| ], | ||
| "tls": { | ||
| "version": "1.2", | ||
| "version_protocol": "TLS" | ||
| }, | ||
| "url": { | ||
| "original": "https://erp.testcloud.com:443/saleInvoice/readBySyskeySIByRoleAllowed/2112140619239361392" | ||
| }, | ||
| "user_agent": { | ||
| "device": { | ||
| "name": "Other" | ||
| }, | ||
| "name": "Chrome", | ||
| "original": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36", | ||
| "os": { | ||
| "full": "Windows 7", | ||
| "name": "Windows", | ||
| "version": "7" | ||
| }, | ||
| "version": "96.0.4664.93" | ||
| } | ||
| }, | ||
| { | ||
| "@timestamp": "2021-12-15T03:35:50.058Z", | ||
| "azure": { | ||
| "frontdoor": { | ||
| "access": { | ||
| "backend_hostname": "samplev6erp.azurewebsites.net:443", | ||
| "cache_status": "CONFIG_NOCACHE", | ||
| "error_info": "NoError", | ||
| "is_received_from_client": true, | ||
| "pop": "SIN", | ||
| "routing_rule_name": "erp", | ||
| "rules_engine_match_names": [], | ||
| "time_taken": "0.064", | ||
| "time_to_first_byte": "0.064" | ||
| }, | ||
| "category": "FrontdoorAccessLog", | ||
| "operation_name": "Microsoft.Network/FrontDoor/AccessLog/Write", | ||
| "resource_id": "/SUBSCRIPTIONS/49D1B571-1CBE-402D-B523-AFEE3C19B64E/RESOURCEGROUPS/WAF-RG/PROVIDERS/MICROSOFT.NETWORK/FRONTDOORS/TESTCLOUD", | ||
| "tracking_reference": "0lWK5YQAAAAAnKnstK4rwSovl+unjuKhoU0lOMzBFREdFMDIxNwBkYjIxMTMyNi1mZTJmLTQwYWYtOTA4My03MGUyYTJmYWRmZjc=" | ||
| } | ||
| }, | ||
| "client": { | ||
| "address": "175.16.199.0", | ||
| "ip": "175.16.199.0", | ||
| "port": 6610 | ||
| }, | ||
| "cloud": { | ||
| "provider": "azure" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.6.0" | ||
| }, | ||
| "event": { | ||
| "category": [ | ||
| "network" | ||
| ], | ||
| "type": [ | ||
| "connection" | ||
| ] | ||
| }, | ||
| "http": { | ||
| "request": { | ||
| "bytes": 1971, | ||
| "method": "GET" | ||
| }, | ||
| "response": { | ||
| "bytes": 637, | ||
| "status_code": 200 | ||
| }, | ||
| "version": "2.0.0.0" | ||
| }, | ||
| "network": { | ||
| "protocol": "HTTPS" | ||
| }, | ||
| "source": { | ||
| "geo": { | ||
| "city_name": "Changchun", | ||
| "continent_name": "Asia", | ||
| "country_iso_code": "CN", | ||
| "country_name": "China", | ||
| "location": { | ||
| "lat": 43.88, | ||
| "lon": 125.3228 | ||
| }, | ||
| "region_iso_code": "CN-22", | ||
| "region_name": "Jilin Sheng" | ||
| } | ||
| }, | ||
| "tags": [ | ||
| "preserve_original_event", | ||
| "azure-frontdoor-access" | ||
| ], | ||
| "tls": { | ||
| "version": "1.2", | ||
| "version_protocol": "TLS" | ||
| }, | ||
| "url": { | ||
| "original": "https://erp.testcloud.com:443/Customer/searchContactList/2107050813256062892" | ||
| }, | ||
| "user_agent": { | ||
| "device": { | ||
| "name": "Other" | ||
| }, | ||
| "name": "Chrome", | ||
| "original": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36", | ||
| "os": { | ||
| "full": "Windows 7", | ||
| "name": "Windows", | ||
| "version": "7" | ||
| }, | ||
| "version": "96.0.4664.93" | ||
| } | ||
| }, | ||
| null | ||
| ] | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.