feat(cloud): Add log reading capabilities to SyncAttempt#786
Merged
Aaron ("AJ") Steers (aaronsteers) merged 4 commits intoSep 16, 2025
Conversation
- Add get_log_text_tail() method for retrieving last N lines of logs - Add get_full_log_text() method for complete log retrieval - Use Config API /v1/attempts/get_for_job endpoint for log access - Handle both structured (LogEvents) and unstructured (LogRead) formats - Fix field names to match Config API schema (events vs logEvents) - Follow existing PyAirbyte patterns for lazy loading and caching Co-Authored-By: AJ Steers <aj@airbyte.io>
Contributor
Original prompt from AJ Steers |
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This PyAirbyte VersionYou can test this version of PyAirbyte using the following: # Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757984248-sync-attempt-log-operations' pyairbyte --help
# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757984248-sync-attempt-log-operations'Helpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
Community SupportQuestions? Join the #pyairbyte channel in our Slack workspace. |
Comment on lines
+185
to
+194
| def get_log_text_tail(self, num_lines: int = 1000) -> str: | ||
| """Return the last N lines of log text for this attempt. | ||
|
|
||
| Args: | ||
| num_lines: Maximum number of lines to return from the end of the logs. | ||
| Defaults to 1000 lines. | ||
|
|
||
| Returns: | ||
| String containing the last N lines of log text, with lines separated by newlines. | ||
| """ |
Member
Author
There was a problem hiding this comment.
This doesn't add anything meaningful. Let's remove this method.
- Address GitHub comment from @aaronsteers - Keep only get_full_log_text() method for complete log retrieval - Simplify SyncAttempt log interface to single method Co-Authored-By: AJ Steers <aj@airbyte.io>
…757984248-sync-attempt-log-operations
…757984248-sync-attempt-log-operations
Aaron ("AJ") Steers (aaronsteers)
merged commit 9cf622f
into
devin/1757543890-sync-attempt-abstraction
15 of 16 checks passed
Aaron ("AJ") Steers (aaronsteers)
deleted the
devin/1757984248-sync-attempt-log-operations
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR targets the following PR:
SyncAttemptabstraction for fetching job attempts #781feat(cloud): Add log reading capabilities to SyncAttempt
Summary
Adds two new methods to the
SyncAttemptclass to enable log retrieval from Cloud sync attempts:get_log_text_tail(num_lines: int = 1000) -> str: Returns the last N lines of log textget_full_log_text() -> str: Returns complete log text for the attemptBoth methods use the existing Config API integration (
_fetch_attempt_info()) and handle both structured (LogEvents) and unstructured (LogRead) log formats returned by the/v1/attempts/get_for_jobendpoint.Review & Testing Checklist for Human
get_full_log_text()doesn't cause memory issuesNotes
_fetch_attempt_info()eventsvslogLines) were verified against the Config API OpenAPI schema_make_config_api_request()pattern with proper noqa commentsLink to Devin run: https://app.devin.ai/sessions/946366cdd71140c88397678ef6fb16fd
Requested by: Aaron ("AJ") Steers (@aaronsteers)