[Enhancement] Windows: Add scheduled task parsing support#19193
[Enhancement] Windows: Add scheduled task parsing support#19193w0rk3r wants to merge 12 commits into
Conversation
w0rk3r
added
the
documentation
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🚀 Benchmarks reportTo see the full report comment with |
andrewkroh
added
the
Team:Security-Windows Platform
|
Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform) |
| --- | ||
| description: Pipeline for parsing Windows Security scheduled task XML. | ||
| processors: | ||
| - script: |
There was a problem hiding this comment.
Instead of parsing this in the ingest node with a custom painless script, would it be possible to use the decode xml processor on the agent? Renames and cleanup could still be done at the ingest node, but it should greatly reduce the size of this script and mitigate the worse cast performance impact.
There was a problem hiding this comment.
we should test it as the xml processor in filebeat has some conventions that might need attention IIRC. I tried to make elastic/elasticsearch#130337 but it is stalled due to the performance of xpath evaluation. I might try to revisit that and see if we can make it happen on the pipelines and if not I'll see what needs to be done to use the filebeat processor instead.
There was a problem hiding this comment.
For now, what action should I take? As the XML processor is also something "more experimental" and the script is tested and restricted to a smaller subset of event codes, should we just continue with this approach? @marc-gr @leehinman
There was a problem hiding this comment.
I'd rather try using filebeat xml processor and have the pipeline make any needed transformations as @leehinman suggested. We can later on switch to the xml decode in the pipeline if we end up merging the ingest processor. The performance hit of such a script is hard to tell and it looks like it would be also a bit harder to maintain than if we do it in filebeat.
TL;DRBuildkite failed before any package tests ran: the post-checkout hook could not merge tested commit Remediation
Investigation detailsRoot CauseThis is a merge conflict during Buildkite checkout/pipeline upload, not a code or test failure. The failing job checked out
The current PR head is already Evidence
Verification
What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
|
✅ All changelog entries have the correct PR link. |
💚 Build Succeeded
History
cc @w0rk3r |
4 participants
Proposed commit message
Summary
Related issue: github.com/elastic/ia-trade-team/issues/892
Windows Security scheduled task events expose high-value task definition metadata primarily as raw XML in
winlog.event_data.TaskContentandwinlog.event_data.TaskContentNew. This makes scheduled-task detection and tuning rely too heavily onTaskName, which is attacker-controlled and easy to masquerade.This PR parses selected scheduled task XML fields into
winlog.scheduled_task.*for Security scheduled task events in the System and Windows forwarded integrations.Why
Windows scheduled tasks are a common mechanism for persistence, remote execution, privilege abuse, and ransomware deployment. Security events already capture useful scheduled task metadata, but most of the task definition is stored as raw XML in
winlog.event_data.TaskContentandwinlog.event_data.TaskContentNew, withTaskNamebeing the only readily queryable field today.That XML contains the behavioral details needed to understand the task: what it runs, when it runs, which account it uses, and whether it is configured in suspicious ways such as hidden execution or elevated run level. Today, those details are not available as structured fields for search, visualization, tuning, or reliable detection logic.
As a result, some detections have to rely on
TaskName, which is attacker-controlled and easy to make look benign. This weakens detection quality and can make detections bypassable.This PR extracts scheduled task metadata into structured
winlog.scheduled_task.*fields. That makes scheduled task activity easier to investigate and enables more durable detections based on what the task actually does, not just what it is called.What changed
This PR adds scheduled task XML parsing for Windows Security events in both the System
securitydata stream and the Windowsforwardeddata stream.The new parsing extracts selected task definition metadata from
TaskContentandTaskContentNewintowinlog.scheduled_task.*, including:The pipeline handles scheduled task creation, update, enable, disable, and delete events. Delete events do not contain task XML, so they continue to populate the scheduled task name from
TaskNameonly.Performance
A dedicated
system/securitypipeline benchmark was added for the scheduled task parsing path. The benchmark uses Security scheduled task events where every document triggers the new XML parser, so it represents the expected worst-case ingest cost for this change.Warm-run results from the same local environment:
This is a worst-case benchmark because every event in the fixture set is a scheduled task event with task XML. In normal Security event streams, the impact should scale with the proportion of events that are scheduled task events carrying
TaskContentorTaskContentNew.Checklist
changelog.ymlfile.How to test this PR locally
Run the System security pipeline tests:
Run the Windows forwarded pipeline tests:
Run the dedicated pipeline benchmark for the scheduled task parsing path:
cd packages/system elastic-package stack up -d --services=elasticsearch elastic-package benchmark pipeline --data-streams security --use-test-samples=false elastic-package stack downScreenshots