Skip to content

Python: Add PDF file support to OpenAI content parser with filename mapping - #1121

Merged
dmytrostruk merged 8 commits into
microsoft:mainfrom
ltwlf:fix/responses-files
Oct 7, 2025
Merged

Python: Add PDF file support to OpenAI content parser with filename mapping#1121
dmytrostruk merged 8 commits into
microsoft:mainfrom
ltwlf:fix/responses-files

Conversation

@ltwlf

@ltwlf ltwlf commented Oct 2, 2025

Copy link
Copy Markdown
Contributor

Description

Resolves: #1082

This PR adds support for PDF files in the OpenAI content parser by implementing proper mapping for application/* media type to OpenAI's file format.

Changes Made

  • Enhanced OpenAI content parser: Added application/* media type handling in _openai_content_parser method
  • Filename support: Implemented filename extraction from additional_properties["filename"]
  • File format mapping: PDFs are now correctly converted to OpenAI’s file format, including filename and file_data properties. Without the filename, OpenAI throws an exception for files.
  • Test coverage: Added comprehensive tests for PDF handling including custom filename scenarios
  • Add pdf samples and fixed broken image samples in multimodal samples

API Usage

# PDF
pdf_content = DataContent(
    uri="data:application/pdf;base64,JVBERi0xLjQ=",
    media_type="application/pdf",
    additional_properties={"filename": "report.pdf"}
)

Breaking Changes
None - this is a purely additive feature that maintains backward compatibility.

Related

Fixes issue #1082 for Python

@markwallace-microsoft markwallace-microsoft added the python Usage: [Issues, PRs], Target: Python label Oct 2, 2025
@ltwlf ltwlf changed the title Python: Add PDF file support to OpenAI Responses content parser with filename mapping Python: Add PDF file support to OpenAI content parser with filename mapping Oct 2, 2025
@ltwlf
ltwlf marked this pull request as ready for review October 2, 2025 14:33
Copilot AI review requested due to automatic review settings October 2, 2025 14:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds PDF file support to the OpenAI content parser by implementing proper handling of application/* media types and filename mapping functionality.

  • Enhanced OpenAI content parser to handle all application/* media types as files
  • Added filename extraction from additional_properties["filename"] with proper fallback behavior
  • Added comprehensive test coverage for PDF handling and edge cases with filename scenarios

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
python/packages/core/agent_framework/openai/_chat_client.py Updated content parser to handle all application/* media types as OpenAI file format with optional filename support
python/packages/core/agent_framework/openai/_responses_client.py Added application/* media type handling for responses with filename mapping
python/packages/core/tests/openai/test_openai_chat_client.py Added comprehensive tests for PDF and document file handling including edge cases
python/samples/getting_started/multimodal_input/openai_chat_multimodal.py Added PDF test function and refactored to use local sample data instead of external requests
python/samples/getting_started/multimodal_input/azure_chat_multimodal.py Added PDF test function and refactored to use local sample data

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread python/packages/core/tests/openai/test_openai_chat_client.py Outdated
Comment thread python/packages/core/agent_framework/openai/_responses_client.py Outdated
Comment thread python/packages/core/agent_framework/openai/_responses_client.py Outdated
@ltwlf

ltwlf commented Oct 2, 2025

Copy link
Copy Markdown
Contributor Author

@dmytrostruk, my fork detached, so I had to co-fork and create a new pull request (old PR #891). I’ve added a PDF to the multimodal samples and fixed the issue where the image sample wasn’t working for me.

If this is okay, I’ll prepare a pull request for .Net as well.

@markwallace-microsoft markwallace-microsoft added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Oct 2, 2025
@markwallace-microsoft

markwallace-microsoft commented Oct 2, 2025

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework/openai
   _chat_client.py2304381%106–107, 111, 129, 139–140, 142, 153, 160, 164, 166–169, 218, 249, 254–268, 275–277, 354, 357, 359, 361, 384–385, 391, 407, 449
   _responses_client.py4096983%141–142, 145–146, 152–153, 156, 163, 195, 204, 216, 246, 274–275, 298, 319, 381, 458, 463, 467–469, 489, 492, 507–508, 512–514, 562, 582–583, 596–597, 613–614, 647, 649, 693, 697, 699, 708, 710, 719–720, 733, 735, 808–814, 831–836, 855, 873, 883, 885, 903–904, 906–908
TOTAL10269168983% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
1258 98 💤 0 ❌ 0 🔥 28.384s ⏱️

@ekzhu
ekzhu enabled auto-merge October 2, 2025 18:32
Comment thread python/packages/core/agent_framework/openai/_chat_client.py Outdated
Comment thread python/samples/getting_started/multimodal_input/azure_chat_multimodal.py Outdated
Comment thread python/samples/getting_started/multimodal_input/azure_responses_multimodal.py Outdated
Comment thread python/samples/getting_started/multimodal_input/openai_chat_multimodal.py Outdated
Comment thread python/samples/getting_started/multimodal_input/azure_responses_multimodal.py Outdated
@ekzhu
ekzhu disabled auto-merge October 2, 2025 23:58
@ltwlf
ltwlf requested a review from eavanvalkenburg October 3, 2025 08:54
ltwlf added 5 commits October 6, 2025 22:03
- Standardized filename extraction logic between chat and responses clients
- Both clients now omit filename when not provided (no default fallback)
- Added Azure Responses API multimodal sample with PDF support
- Cleaned up Azure Chat sample to focus on supported features only
- Fixed test comment placement for better code documentation
- Updated README with clear API capability differences
- Refactor image and PDF handling in `azure_chat_multimodal.py` and `openai_chat_multimodal.py` to use new utility functions.
- Add `load_sample_pdf` and `create_sample_image` functions for better test asset management.
- Remove redundant code for creating sample images and PDFs.
- Introduce a sample PDF file in `sample_assets` for testing purposes.
@ltwlf
ltwlf force-pushed the fix/responses-files branch from 8ac0997 to 887445a Compare October 6, 2025 20:09
@dmytrostruk
dmytrostruk added this pull request to the merge queue Oct 6, 2025
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 6, 2025
@dmytrostruk
dmytrostruk enabled auto-merge October 7, 2025 16:59
@dmytrostruk
dmytrostruk added this pull request to the merge queue Oct 7, 2025
Merged via the queue into microsoft:main with commit 8bb1266 Oct 7, 2025
25 checks passed
@ltwlf
ltwlf deleted the fix/responses-files branch October 7, 2025 18:10
ReubenBond pushed a commit to ReubenBond/agent-framework that referenced this pull request Oct 28, 2025
…apping (microsoft#1121)

* Add mapping for application media type in OpenAI responses client

* Enhance multimodal input samples: Add PDF testing functionality and fix image sample

* Standardize filename handling and add multimodal samples

- Standardized filename extraction logic between chat and responses clients
- Both clients now omit filename when not provided (no default fallback)
- Added Azure Responses API multimodal sample with PDF support
- Cleaned up Azure Chat sample to focus on supported features only
- Fixed test comment placement for better code documentation
- Updated README with clear API capability differences

* Enhance multimodal input samples with image and PDF handling

- Refactor image and PDF handling in `azure_chat_multimodal.py` and `openai_chat_multimodal.py` to use new utility functions.
- Add `load_sample_pdf` and `create_sample_image` functions for better test asset management.
- Remove redundant code for creating sample images and PDFs.
- Introduce a sample PDF file in `sample_assets` for testing purposes.

* Fix formatting in OpenAI chat client

---------

Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
arisng pushed a commit to arisng/agent-framework that referenced this pull request Feb 2, 2026
…apping (microsoft#1121)

* Add mapping for application media type in OpenAI responses client

* Enhance multimodal input samples: Add PDF testing functionality and fix image sample

* Standardize filename handling and add multimodal samples

- Standardized filename extraction logic between chat and responses clients
- Both clients now omit filename when not provided (no default fallback)
- Added Azure Responses API multimodal sample with PDF support
- Cleaned up Azure Chat sample to focus on supported features only
- Fixed test comment placement for better code documentation
- Updated README with clear API capability differences

* Enhance multimodal input samples with image and PDF handling

- Refactor image and PDF handling in `azure_chat_multimodal.py` and `openai_chat_multimodal.py` to use new utility functions.
- Add `load_sample_pdf` and `create_sample_image` functions for better test asset management.
- Remove redundant code for creating sample images and PDFs.
- Introduce a sample PDF file in `sample_assets` for testing purposes.

* Fix formatting in OpenAI chat client

---------

Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: OpenAI Responses Client: Missing application/* content type handling causes API error

6 participants