Skip to content

Arp demonstration - #21

Open
tychon1 wants to merge 960 commits into
mainfrom
Arp-Demonstration
Open

Arp demonstration#21
tychon1 wants to merge 960 commits into
mainfrom
Arp-Demonstration

Conversation

@tychon1

@tychon1 tychon1 commented Nov 15, 2023

Copy link
Copy Markdown
Collaborator

What does this PR do?

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

kgeller and others added 30 commits October 2, 2023 11:26
)

* Add configuration option for tld for guardduty, inspector, securityhub_findings and securityhub_insights httpjson configs

* changelog

* switch to enhancement

* system test config for tld for securityhub_findings datastream

* syncing tld in docker-compose  with the system test config
…c#8039)

Bumps [github.com/elastic/elastic-package](https://github.com/elastic/elastic-package) from 0.87.1 to 0.88.0.
- [Release notes](https://github.com/elastic/elastic-package/releases)
- [Changelog](https://github.com/elastic/elastic-package/blob/main/.goreleaser.yml)
- [Commits](elastic/elastic-package@v0.87.1...v0.88.0)

---
updated-dependencies:
- dependency-name: github.com/elastic/elastic-package
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

* Filter out validation errors about filters in dashboards
* Filter out validation errors about having queries but no filters in dashboards
* Filter out more validation errors about filters in dashboards
* Add validation filter for osquery_manager
* Replace unexistent 8.10.0 version with 8.10.1

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Co-authored-by: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
* [synthetics] Update Synthetics integration minor version for changes starting from Kibana v8.11. 
* Break out Synthetics Dashboards
…eam (elastic#7356)

* [Azure][Storage_account] Add dimension to the storage account datastream and update Kibana version.
…nt datastream (elastic#7488)

* [Azure Storage Account] Add metric_type metadata to the storage_account  datastream.
…eam inputs (elastic#8014)

* [cisco_ise] Add filestream fields

* [cisco_nexus] Add new filestream fields

* [cisco_ise] Fix docs

* [f5_bigip] Add new filestream fields

* Fix CI errors

* [fortinet_fortimail] Add new filestream fields

* [fortinet_fortimanager] Add new filestream fields

* [juniper_srx] Add new filestream fields

* [keycloak] Add filestream fields

* [mysql_enterprise] Add new filestream fields

* [hid_bravura_monitor] Add filestream fields

* [sysmon_linux] Add new filestream fields

* [trendmicro] Add new filestream fields

* running system tests and generating new readme

---------

Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
* Initial Release for Wiz

* Update the changelog entry

* Resolve review comments

* Resolved review comments. Added pagination in the system test.
…stic#8076)

* [Security Rules] Update security rules package to v8.10.4-beta.1

* Add changelog entry for 8.10.4-beta.1

* Update packages/security_detection_engine/manifest.yml
…ies.effective` (elastic#8051)

* updating journald to ecs 8.10 to pull in new process.thread.capabilities.effective field

* regenerated

* accidental deletion reverted

* putting back min version required and updating changelog
Modified the field definitions to reference ECS where possible and remove
invalid field attributes.

[git-generate]
go run github.com/andrewkroh/go-examples/ecs-update@v0.0.0-20231003061136-3ff9278e0277 -skip-format -fields-yml-use-ecs -fields-yml-cleanup-attrs packages/zeek
…ic#8071)

* Add event.code

* update pr url

* Update packages/cisco_ise/changelog.yml

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

* Linter found a missing space

* Added event.code field to mapping and build it, so docs are updated

---------

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
* Add Azure rule templates

* Adding additional rules and bumping version

* Updated rules

* Add manifest and bump
… for filestream inputs (elastic#8068)

* [docker] Add new filestream fields

* [istio] Add filestream fields

* [kubernetes] Add filestream fields

* [nginx_ingress_controller] Add filestream fields
…ns (elastic#8016)

* Migrate Prometheus Server Overview dashboard to lens

* Update changelog
…tic#8089)

It fixes the processing of event 313005 which contains a protocol IANA number inside the original IP payload that was not being parsed.
Add condition in rename processor from message to event.original to check if the field already exists
* Packaging network beaconing (with serverless support)
For fields that exist in ECS use 'external: ecs' in the field definition.

Remove attributes from field definitions that have no purpose (like footnote, level, title, etc).

Fix typo for description attribute in some field definitions.

Remove duplicate definitions of fields within the same data stream.

Fix dotted YAML key in manifest.yml.

[git-generate]
cd packages/elastic_agent
go run github.com/andrewkroh/fydler@90f7b627e -fix -a useecs,invalidattribute,unknownattribute **/fields/*.yml
perl -p -i -e 's/descripion:/description:/g' **/fields/*.yml
elastic-package format
elastic-package changelog add --link elastic#8079 --type enhancement --next minor --description "Modified the field definitions to reference ECS where possible and remove invalid field attributes."
Use 'external: ecs' where possible.

Set 'type: group' instead of 'type: object' on field groups.

Remove unnecessary field attributes (like group, footnote).

Add 'type' to fields where it was missing. I think an explicit type will be better than depending on defaults
being injected elsewhere.

Resolve duplicate 'type: group/object'
[git-generate]
cd packages/box_events
go run github.com/andrewkroh/fydler@e90f9031ec48 -fix **/fields/*
elastic-package format





[git-generate]
elastic-package changelog add --link elastic#8097 --type enhancement --next minor --description "Modified the field definitions to reference ECS where possible and remove invalid field attributes."

* [box_events] - Remove format attribute

[git-generate]
go run github.com/andrewkroh/fydler@e90f9031ec48 -fix -a unknownattribute packages/box_events/**/fields/*.yml
…ens (elastic#8058)

* 8057 Containerd Overview lens migration initial commit

* update changelog entry
robbavey and others added 30 commits October 31, 2023 20:39
This PR adds additional drilldown dashboards on input, filter and output plugin in order to to add contextual information about where a plugin was defined. This is important, as it allows users to quickly find out which of many plugins
of the same type are being referenced in the dashboards, without needing to rewrite long and complex pipeline configurations to add identifiers to each of their plugin definitions.

To do this, the CEL code is augmented to retrieve pipeline information from the /pipelines/#{pipeline_name} endpoint, which provides information about each of the pipeline graphs.

Additionally, this PR adds CEL code to pull metrics data from the pipeline graph, and make this available to plugin specific drilldowns, with dashboards for logstash-filter-dissect, logstash-filter-grok and logstash-output-elasticsearch provided here.

Additionally, this commit includes the necessary field mappings for the dissect and grok filters and elasticsearch output, and drilldown dashboards for dissect, grok and the elasticsearch output and the necessary links from the single pipeline view to enable access to these dashboards.

This also includes improvements to dashboard organization and consistency, ensuring
that dashboards are all organized in the same way with "events received" always being
shown before "events emitted" and organized vertically for improved comparisons.

There are also some updates made for consistent labelling of visualizations.
The previous table was incorrect. Table data comes from MS-SAMR: Security
Account Manager (SAM) Remote Protocol (Client-to-Server) version 46.0[1],
2.2.1.12 USER_ACCOUNT Codes.

[1]https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SAMR/%5bMS-SAMR%5d-230828.docx
elastic#8322)

In the `firewall_match` pipeline of the Crowdstrike integration's
Falcon data stream, prefer `ImageFileName` over `CommandLine` as the
source of the value for `process.executable` when both are populated,
as `ImageFileName` should provide the more complete file name.

---------

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
… field to `client.ip` (elastic#8304)

* fix pipeline test and rename header_forwarder field to client.ip

* update changelog.yml

* address review comments

* address review comments
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
* CloudFormation version hardening

* add pr number

* fix changelog
* make spring boot ga

* update pr link in changelog
* address post-merge feedback in elastic#8240

* add link to this pr
…ields (elastic#8350)

It fixes an error when parsing EvidenceString fields looking for providers that fill threat.indicator.provider list later in the pipeline. The gsub processor was looking for a specific pattern but didn't discard the whole field when it didn't match.
…c#8354)

Several dissect processors have been replaced by a kv processor that makes the processing of ip_flow events less restrictive, so now ICMP events are also processed as well as TCP/UDP ones.
…lastic#8379)

* migrate app state overview dashboard visualizations to lens

* update changelog
…#8386)

* Add system test for aws-s3 input

* Fix fields and bump version
Improve clarity of CEL code:
* make evaluation step clearly defined
* remove redundant as macro use
* use string for RFC 3339 timestamp formatting
* use more vertical whitespace

Reduce repeated work:
* hoist request query parsing out of struct

Add fallback for case where request URL in response has no startTime.

Add mappings for missing fields:
* o365.audit.AdditionalInfo
* o365.audit.AppAccessContext
… action. (elastic#8384)

* Fix handling of security events without dhost and with action.

* Add changelog PR

* Trim values
…elastic#8382)

* migrate client ip count timeline visualization to lens.

* update changelog.yml
…porting (elastic#8346)

* Fix Jenkinsfile to use the json files for pipeline benchmarks reporting

* Change extension in stashBenchmarkResults
…elastic#8383)

* migrate client ip count timeline` visualization to lens.

* update changelog.yml
…astic#8388)

* Add system tests and fix fields

* Address PR comments

* Fix new line

* Replace the proxy docs in test config

* add new line in test file
…ic#8387)

Bumps [github.com/elastic/package-registry](https://github.com/elastic/package-registry) from 1.21.0 to 1.22.0.
- [Release notes](https://github.com/elastic/package-registry/releases)
- [Changelog](https://github.com/elastic/package-registry/blob/main/CHANGELOG.md)
- [Commits](elastic/package-registry@v1.21.0...v1.22.0)

---
updated-dependencies:
- dependency-name: github.com/elastic/package-registry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.