Kafka improve logging

[CarlosGamero]

Summary by CodeRabbit

• Tests

  • Removed stream control flow validation tests

• Chores

  • Enhanced commit operation logging with detailed status messages
  • Optimized message batch processing
  • Improved stream handling during batch operations

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Removed stream pause/resume behavior from batch processing in AbstractKafkaConsumer and deleted its associated test, while enhancing commit operation logging with structured messages before and after commit attempts.

Changes

Cohort / File(s)	Change Summary
Stream pause/resume removal and logging enhancements packages/kafka/lib/AbstractKafkaConsumer.spec.ts, packages/kafka/lib/AbstractKafkaConsumer.ts	Deleted test verifying pause/resume sequence during batch processing. Removed consumerStream pause before and resume after batch processing in handleSyncStreamBatch. Simplified empty-batch commit handling to single-line return. Enhanced commitMessage() with structured per-message logging: "Trying to commit message" before attempt and success/failure messages after.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Pause/resume removal rationale: Verify the behavioral implications and confirm that removing stream pause/resume does not cause unintended message processing side effects
• Test deletion: Confirm test deletion aligns with intended behavior change
• Logging structure: Review new commitMessage() logging for consistency with existing patterns

Possibly related PRs

• Stop stream accepting messages when consuming in process #376 — Directly modifies AbstractKafkaConsumer.handleSyncStreamBatch and consumer stream pause/resume behavior; represents the inverse change pattern
• Waiting for a batch to be processed before accepting more messages #371 — Modifies handleSyncStreamBatch implementation and batch-processing flow within AbstractKafkaConsumer
• use for await instead of stream.on #360 — Modifies stream consumption patterns and pause/resume behavior in AbstractKafkaConsumer

Suggested reviewers

• kibertoad
• kjamrog

Poem

🐰 The streams now flow without their pauses,
No rests between the batch's causes,
With logging bright for each commit's fate,
The consumer hops—no longer late!
Pause and resume? They've said goodbye,
Cleaner flow 'neath the Kafka sky. ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Kafka improve logging' is directly related to the main changes: enhanced logging in commitMessage() with structured details and simplified commit() logging.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch kafka_improve_logging

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 61e2b57 and 5b8c49f.

📒 Files selected for processing (2)

• packages/kafka/lib/AbstractKafkaConsumer.spec.ts (0 hunks)
• packages/kafka/lib/AbstractKafkaConsumer.ts (2 hunks)

💤 Files with no reviewable changes (1)

• packages/kafka/lib/AbstractKafkaConsumer.spec.ts

🔇 Additional comments (2)

packages/kafka/lib/AbstractKafkaConsumer.ts (2)

435-436: LGTM: Cleaner empty batch handling.

The single-line return for empty batches is more concise and maintains the same behavior.

---

445-456: LGTM: Enhanced commit observability.

The structured logging with topic, offset, and timestamp provides valuable debugging context. Using debug level is appropriate for high-frequency commit operations, and no sensitive data is exposed.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}