diff --git a/README.md b/README.md index 63eb0e7..06bc753 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ This App is designed to work across multiple GitHub data sources however not all The GitHub App for Splunk is designed to work with the following data sources: * [GitHub Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from GitHub Enterprise Cloud. -* [Github.com Webhooks](./docs/github_webhooks.MD): A select set of webhook events like Push, PullRequest, and Repo. +* [Github.com Webhooks](./docs/github_webhooks.MD): A select set of webhook events like Push, PullRequest, Code Scanning and Repo. * [Github Enterprise Server Syslog Forwarder](./docs/ghes_syslog_setup.MD): Audit and Application logs from Github Enterprise Server. * [Github Enterprise Collectd monitoring](./docs/splunk_collectd_forwarding_for_ghes.MD): Performance and Infrastructure metrics from Github Enterprise Server. @@ -37,6 +37,9 @@ There is an *Integration Overview* dashboard listed under *Dashboards* that allo
Expand for screenshots +#### Code Scanning Alerts + ![Code Scanning Dashboard](./docs/images/code_scanning_dashboard.png) + #### Audit Log Dashboard ![Audit Log Dashboard](./docs/images/9F8E9A89-1203-4C0A-B227-C2FD1E17C8B0.jpg) diff --git a/docs/github_webhooks.MD b/docs/github_webhooks.MD index 6c844c4..140d3ae 100644 --- a/docs/github_webhooks.MD +++ b/docs/github_webhooks.MD @@ -64,4 +64,10 @@ Once that is complete and webhooks are triggering, you'll want to update the mac Pull request reviews Pull request review submitted, edited, or dismissed. + +GitHub::CodeScanning +Code scanning alerts +Alerts identified by CodeQL and other 3rd party/OSS scanning tools. + + diff --git a/docs/images/code_scanning_dashboard.png b/docs/images/code_scanning_dashboard.png new file mode 100644 index 0000000..92135b9 Binary files /dev/null and b/docs/images/code_scanning_dashboard.png differ diff --git a/github_app_for_splunk/README.md b/github_app_for_splunk/README.md index 34d6b76..d6c1918 100644 --- a/github_app_for_splunk/README.md +++ b/github_app_for_splunk/README.md @@ -1,13 +1,13 @@ # GitHub App for Splunk -The GitHub App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give GitHub Admins and platform owners immediate visibility into GitHub. +The GitHub App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give GitHub Admins, platform owners, and Security Engineers immediate visibility into GitHub. This App is designed to work across multiple GitHub data sources however not all all required. You may choose to only collect a certain set of data and the parts of this app that utilize that set will function, while those that use other data sources will not function correctly, so please only use the Dashboards that relate to the data you are collecting. The GitHub App for Splunk is designed to work with the following data sources: * [GitHub Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from GitHub Enterprise Cloud. -* [Github.com Webhooks]((./docs/github_webhooks.MD)): A select set of webhook events like Push, PullRequest, and Repo. +* [Github.com Webhooks]((./docs/github_webhooks.MD)): A select set of webhook events like Push, PullRequest, Repo, and Code Scanning alerts. * [GitHub Enterprise Server Syslog Forwarder](https://docs.github.com/en/enterprise-server@3.0/admin/user-management/monitoring-activity-in-your-enterprise/log-forwarding): Audit and Application logs from GitHub Enterprise Server. * [GitHub Enterprise Collectd monitoring](./docs/splunk_collectd_forwarding_for_ghes.MD): Performance and Infrastructure metrics from GitHub Enterprise Server. diff --git a/github_app_for_splunk/default/data/ui/nav/default.xml b/github_app_for_splunk/default/data/ui/nav/default.xml index 6837a78..09c19a4 100644 --- a/github_app_for_splunk/default/data/ui/nav/default.xml +++ b/github_app_for_splunk/default/data/ui/nav/default.xml @@ -8,8 +8,9 @@ - + + diff --git a/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml b/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml new file mode 100644 index 0000000..39e236e --- /dev/null +++ b/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml @@ -0,0 +1,192 @@ +
+ + + + `github_webhooks` (eventtype="GitHub::CodeScanning" OR eventtype="GitHub::Push") | eval action='action', tool=if(isnotnull('alert.tool.name'),'alert.tool.name','unknown'), repository=if(isnotnull('repository.name'),'repository.name','unknown'), severity=if(isnotnull('alert.rule.security_severity_level'),'alert.rule.security_severity_level','none'), create_time=if(isnotnull('alert.created_at'),'alert.created_at','unknown'), received_time='_time', alert_url=if(isnotnull('alert.html_url'),'alert.html_url','unknown'), eventtype='eventtype', created=strptime(create_time, "%Y-%m-%dT%H:%M:%S%Z"), duration=received_time - created, duration_str=tostring(avg(duration), "duration") + + $timeTkn.earliest$ + $timeTkn.latest$ + 1 + +
+ + + + -24h@h + now + + + + + tool + tool + " + " + + | table tool | dedup tool + + All + * + * + + + + All + * + * + " + " + , + repository + repository + + | dedup repository | table repository + + +
+ + + + Created + + | search tool=$tool_name$ repository=$repoTkn$ action="created" | stats count + + + + + + + + + + Fixed + + | search tool=$tool_name$ repository=$repoTkn$ action="fixed" | stats count + + + + + + + + + Reopened + + | search tool=$tool_name$ repository=$repoTkn$ action="reopened" | stats count + + + + + + + + + + + Alert Found/Fixed Ratio + + | search tool=$tool_name$ repository=$repoTkn$ (action=created OR action=fixed) +| timechart count(_raw) by action +| accum created +| accum fixed +| rename created as "Found" +| rename fixed as "Fixed" + + + + + + + + + + + Commit/Alert Ratio + + | search (eventtype="GitHub::Push" repository=$repoTkn$) OR ((action=created OR action=reopened) tool=$tool_name$ repository=$repoTkn$ ) +| timechart count(_raw) by eventtype +| accum "GitHub::Push" +| accum "GitHub::CodeScanning" +| rename GitHub::Push as "Pushes" +| rename GitHub::CodeScanning as "Code Scanning Alerts" + + + + + + + + + + + + + + + New Alerts by Tool + + | search tool=$tool_name$ repository=$repoTkn$ (action=created OR action=appeared_in_branch) | timechart count(_raw) by tool + + + + + + + + + + + + + Fixed Alerts + + | search (action=fixed OR action=closed_by_user) repository=$repoTkn$ tool=$tool_name$ +| table repository, tool, alert_url,duration_str +| rename repository AS "Repository" duration_str AS "Time to Resolution",tool AS "Tool", alert_url AS "Alert URL" +| sort -"Time to Resolution" + + + +
+ + + + + + Alerts by Severity + + | search (action=created OR action=reopened) repository=$repoTkn$ tool=$tool_name$ | chart usenull=f count over repository by severity + + + + + + + + + + + + + + +
+ + + + Alerts by Repo + + | search (action=created OR action=reopened) repository=$repoTkn$ tool=$tool_name$| chart usenull=f count over repository by tool + + + + + + + + + +
+ + +
\ No newline at end of file diff --git a/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml b/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml index a19e82f..c9ab484 100644 --- a/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml +++ b/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml @@ -1,8 +1,8 @@
- + - index="github_webhook" alert.created_at=* | eval reason=if(isnotnull('alert.affected_package_name'),'alert.affected_package_name','alert.rule.name'), id=if(isnotnull('alert.external_identifier'),'alert.external_identifier','alert.rule.id'), severity=if(isnotnull('alert.severity'),'alert.severity','alert.rule.security_severity_level'), type=if(isnotnull('alert.external_identifier'),"vulnerability alert","code scanning alert") | stats latest(action) as status, earliest(alert.created_at) as created_at by repository.name, reason, id, type, severity | eval age = toString(round(now() - strptime(created_at, "%Y-%m-%dT%H:%M:%S")),"Duration") + `github_webhooks` alert.created_at=* | eval reason=if(isnotnull('alert.affected_package_name'),'alert.affected_package_name','alert.rule.name'), id=if(isnotnull('alert.external_identifier'),'alert.external_identifier','alert.rule.id'), severity=if(isnotnull('alert.severity'),'alert.severity','alert.rule.security_severity_level'), type=if(isnotnull('alert.external_identifier'),"Dependabot Alert","Code Scanning Alert") | stats latest(action) as status, earliest(alert.created_at) as created_at by repository.name, reason, id, type, severity | eval age = toString(round(now() - strptime(created_at, "%Y-%m-%dT%H:%M:%S")),"Duration") $timeTkn.earliest$ $timeTkn.latest$ diff --git a/github_app_for_splunk/default/data/ui/views/webhook_config.xml b/github_app_for_splunk/default/data/ui/views/webhook_config.xml index 80d4921..4a41155 100644 --- a/github_app_for_splunk/default/data/ui/views/webhook_config.xml +++ b/github_app_for_splunk/default/data/ui/views/webhook_config.xml @@ -64,6 +64,12 @@ Pull request reviews Pull request review submitted, edited, or dismissed. + + GitHub::CodeScanning + Code Scanning alerts + Alert created, fixed, reopened, appeared in branch, closed by user, or reopened by user. + + diff --git a/github_app_for_splunk/default/data/ui/views/welcome_page.xml b/github_app_for_splunk/default/data/ui/views/welcome_page.xml index 183c0d5..ecadfb6 100644 --- a/github_app_for_splunk/default/data/ui/views/welcome_page.xml +++ b/github_app_for_splunk/default/data/ui/views/welcome_page.xml @@ -39,6 +39,9 @@
  • The "Enterprise Server Monitor" drop down has several dashboards that report on the health and performance of your GHES environment
  • "Audit" drop down works for GHES as well as GitHub.com audit logs
  • The Alerts menu item contains all GitHub recommended alerts
    • +
  • Audit Log Activity provides a highlevel overview of what activity is going on in GitHub.
    • +
  • Repository Audit lets you review changes to individual or groups of repositories
    • +
  • User Change Audit is the best place to review actions taken by or made to individual users.

    • @@ -55,13 +58,12 @@ }
    -

    Audit and Security

    +

    GitHub Advanced Security

    - Security users will probably be most interested in the Audit dashboards + Open Source repositories and customers of GitHub Advanced Security have access to application security tooling such as Code Scanning, Secret Scanning, and Dependency Review.

      -
    1. Audit Log Activity provides a highlevel overview of what activity is going on in GitHub.
      2. -
    2. Repository Audit lets you review changes to individual or groups of repositories
      3. -
    3. User Change Audit is the best place to review actions taken by or made to individual users.
      4. +
    4. The Advanced Security Overview dashboard gives insight into the security posture of your GitHub Organization
      5. +
    5. The Code Scanning dashboard gives you access to alerts created by Code Scanning within your Organization

    @@ -85,7 +87,7 @@ GitHub has several ways to collect data from their services depending on your needs. Information is available within the App on how to collect different types of data from GitHub:
    1. Audit Log data is available through a Splunk Add-On
      2. -
    2. Rich commit and pull request data is available through GitHub Webhooks
      3. +
    3. Rich commit, pull request, and Code Scanning data is available through GitHub Webhooks

    diff --git a/github_app_for_splunk/default/eventtypes.conf b/github_app_for_splunk/default/eventtypes.conf index 71e1e06..aca0a21 100644 --- a/github_app_for_splunk/default/eventtypes.conf +++ b/github_app_for_splunk/default/eventtypes.conf @@ -27,3 +27,6 @@ search = `github_webhooks` action IN ("created","edited","moved","deleted") "pr [GitHub::Workflow] search = `github_webhooks` action IN ("queued","created","started","completed") workflow_job.id=* + +[GitHub::CodeScanning] +search = `github_webhooks` action IN ("appeared_in_branch", "closed_by_user", "created", "fixed", "reopened", "reopened_by_user") "alert.created_at"=* \ No newline at end of file diff --git a/github_app_for_splunk/default/props.conf b/github_app_for_splunk/default/props.conf index 4edc247..b6fcbb1 100644 --- a/github_app_for_splunk/default/props.conf +++ b/github_app_for_splunk/default/props.conf @@ -29,7 +29,7 @@ disabled = false pulldown_type = 1 [github_json] -DATETIME_CONFIG = +DATETIME_CONFIG = CURRENT LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true TRUNCATE = 250000