Skip to content

Fix #5614: Handle UTF-8 BOM in EPW files#5629

Merged
jmarrec merged 2 commits into
NatLabRockies:developfrom
wnykuang:agent/fix-epw-utf8-bom
Jul 15, 2026
Merged

Fix #5614: Handle UTF-8 BOM in EPW files#5629
jmarrec merged 2 commits into
NatLabRockies:developfrom
wnykuang:agent/fix-epw-utf8-bom

Conversation

@wnykuang

@wnykuang wnykuang commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix EpwFile cannot process epw file #5614
  • Strip a leading UTF-8 byte order mark before parsing an EPW LOCATION header.
  • Add a regression test proving BOM-prefixed EPW content remains loadable.

Root cause

EpwFile::parse passed the first line through unchanged, while parseLocation requires the first comma-delimited field to equal LOCATION. For UTF-8-with-BOM files, that field begins with the bytes EF BB BF, so a valid header was reported as missing its LOCATION specifier.

The fix only removes a UTF-8 BOM at the beginning of the first line. Existing validation remains unchanged, and this does not attempt to decode UTF-16 or UTF-32 files.

User impact

EPW files saved as UTF-8 with BOM, including the file attached to #5614, can now be processed normally.

Validation

  • Added Filetypes.EpwFile_UTF8BOM and observed it fail before the implementation change and pass afterward.
  • Ran all 25 Filetypes.EpwFile* tests successfully.
  • Re-ran the original 2.5 MB EPW attachment from EpwFile cannot process epw file #5614 and confirmed the parsing failure no longer occurs.
  • Ran git diff --check successfully.

Fixes #5614.

Strip a leading UTF-8 byte order mark before validating the LOCATION header. Add a regression test covering BOM-prefixed EPW content.\n\nFixes NatLabRockies#5614.
@wnykuang wnykuang changed the title Fix EPW parsing for UTF-8 BOM Fix #5614: Handle UTF-8 BOM in EPW files Jul 11, 2026
@wnykuang
wnykuang marked this pull request as ready for review July 11, 2026 21:01
@wnykuang
wnykuang marked this pull request as draft July 11, 2026 21:04
@wnykuang
wnykuang marked this pull request as ready for review July 11, 2026 21:06

namespace openstudio {

constexpr std::string_view UTF8_BOM{"\xEF\xBB\xBF", 3};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Not my prefered style of declaration, but it's valid and defensive (would work with a null byte in the middle too)

Comment thread src/utilities/filetypes/EpwFile.cpp
@jmarrec
jmarrec merged commit 79253ab into NatLabRockies:develop Jul 15, 2026
3 of 11 checks passed
@wnykuang
wnykuang deleted the agent/fix-epw-utf8-bom branch July 15, 2026 18:07
@jmarrec

jmarrec commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Ah shoot, since CI is failing due to BCL tests, I didn't notice that Windows x64 was failing the new test.

https://github.com/NatLabRockies/OpenStudio/actions/runs/29307997109/job/87338834472#step:23:9224

  [ RUN      ] Filetypes.EpwFile_UTF8BOM
  [openstudio.EpwFile] <1> Insufficient weather data on line 8769 of EPW file ''
  D:\a\OpenStudio\OpenStudio\OpenStudio\src\utilities\filetypes\test\EpwFile_GTest.cpp(60): error: Value of: epwFile
    Actual: false
  Expected: true

@wnykuang

jmarrec added a commit that referenced this pull request Jul 16, 2026
Hotfix #5629 - read_as_string should open as binary
@wnykuang

Copy link
Copy Markdown
Contributor Author

@jmarrec thanks for bringing it up, let me try to reproduce and fix it.

@jmarrec

jmarrec commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

I fixed it in #5633 already

@wnykuang

Copy link
Copy Markdown
Contributor Author

I fixed it in #5633 already

Thanks let me close it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EpwFile cannot process epw file

2 participants