feat(loki.source.syslog): support raw format#5140
Conversation
|
💻 Deploy preview deleted (feat(loki.source.syslog): support raw format). |
🔍 Dependency Reviewgithub.com/leodido/go-syslog/v4 v4.2.0 -> v4.3.0 — ✅ Safe
No code changes suggested. Notes
|
internal/component/loki/source/syslog/internal/syslogtarget/syslogparser/rawparser.go
Fixed
Show fixed
Hide fixed
There was a problem hiding this comment.
Pull request overview
This PR adds support for a raw syslog format to consume non-RFC5424 compliant logs such as CEF (Common Event Format). The raw format skips syslog message parsing except for framing detection (octet counting or non-transparent).
Key changes:
- Added new
rawsyslog format option that disables structured syslog parsing - Implemented raw message parser that detects framing type and extracts priority/severity when present
- Added configuration validation to prevent incompatible options when using raw format
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
internal/loki/promtail/scrapeconfig/config.go |
Added SyslogFormatRaw constant and SyslogRawFormatOptions struct with delimiter configuration |
internal/component/loki/source/syslog/config/config.go |
Added SyslogFormat validation, marshaling methods, and RawFormatOptions with delimiter support |
internal/component/loki/source/syslog/types.go |
Updated ListenerConfig to support raw format options and added validation for format-specific fields |
internal/component/loki/source/syslog/internal/syslogtarget/transport.go |
Modified TCP/UDP transports to handle raw format with custom delimiters |
internal/component/loki/source/syslog/internal/syslogtarget/syslogtarget.go |
Added handleMessageRaw method and converted message handling to type switch |
internal/component/loki/source/syslog/internal/syslogtarget/syslogparser/rawparser.go |
Implemented raw parser with octet counting and non-transparent framing detection |
internal/converter/internal/promtailconvert/internal/build/syslog.go |
Updated converter to support raw format and options |
docs/sources/reference/components/loki/loki.source.syslog.md |
Added documentation for raw format and configuration options |
go.mod, dependency-replacements.yaml |
Updated go-syslog dependency to v4.3.0 |
internal/component/loki/source/syslog/internal/syslogtarget/transport.go
Show resolved
Hide resolved
internal/component/loki/source/syslog/internal/syslogtarget/transport.go
Outdated
Show resolved
Hide resolved
|
Nice work with the tests! I think some CI issues and a few copilot comments may be relevant. |
Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>
4ced942 to
1ab7ab5
Compare
internal/component/loki/source/syslog/internal/syslogtarget/syslogtarget_test.go
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
internal/component/loki/source/syslog/internal/syslogtarget/syslogparser/syslogparser.go
Outdated
Show resolved
Hide resolved
This is a followup PR to a [previous PR](#5140) to extend `loki.source.syslog` component to enable parsing IOS-specific syslog fields. This PR exposes existing cisco-specific go-syslog [parser options](https://github.com/leodido/go-syslog/tree/develop/rfc3164#cisco-device-configuration) to component arguments.
This is a followup PR to a [previous PR](#5140) to extend `loki.source.syslog` component to enable parsing IOS-specific syslog fields. This PR exposes existing cisco-specific go-syslog [parser options](https://github.com/leodido/go-syslog/tree/develop/rfc3164#cisco-device-configuration) to component arguments.
This is a followup PR to a [previous PR](#5140) to extend `loki.source.syslog` component to enable parsing IOS-specific syslog fields. This PR exposes existing cisco-specific go-syslog [parser options](https://github.com/leodido/go-syslog/tree/develop/rfc3164#cisco-device-configuration) to component arguments.
This is a followup PR to a [previous PR](#5140) to extend `loki.source.syslog` component to enable parsing IOS-specific syslog fields. This PR exposes existing cisco-specific go-syslog [parser options](https://github.com/leodido/go-syslog/tree/develop/rfc3164#cisco-device-configuration) to component arguments.
PR Description
This PR adds support of
rawformat to consume non-RFC5424 compliant logs such as CEF.The format skips syslog message parsing, except non-transparent/octet count framing detection.
Which issue(s) this PR fixes
Fixes #1711
Notes to the Reviewer
PR Checklist
BEGIN_COMMIT_OVERRIDE
feat(loki.source.syslog): Support raw format (#5140)
END_COMMIT_OVERRIDE