fix: improve Chinese date parsing and handling - #359
Merged
Conversation
1. Added support for more flexible Chinese date formats (missing 月/ 日 suffixes) 2. Fixed potential conflict between size extraction and year-month patterns 3. Added test case to verify size extraction doesn't shadow year-month patterns 4. Enhanced regex patterns to support more date format variations 5. Added checks in SizeExtractor to respect previously matched spans The changes improve Chinese NLP parsing by: - Making common date patterns like "2025年12" (without 月) and "2025年12 月30" (without 日) work - Preventing false positives where size patterns could incorrectly match year components - Supporting more Chinese variations like "年份" suffix and different separators - Adding proper span checking to maintain intent integrity Log: Improved Chinese date parsing with more flexible formats and better conflict resolution Influence: 1. Test various Chinese date formats with and without 月/日 suffixes 2. Verify mixed queries like "2025-5的音乐" correctly parse as date + filetype 3. Test boundary cases like "50KB" vs "2025年5月" 4. Verify all supported date separators (年./-) work correctly fix: 改进中文日期解析和处理功能 1. 添加对更灵活中文日期格式的支持（支持省略月/日后缀） 2. 修复大小提取与年月模式之间的潜在冲突 3. 添加测试用例验证大小提取不会遮盖年月模式 4. 增强正则表达式以支持更多日期格式变化 5. 在SizeExtractor中添加检查以尊重先前匹配的文本片段 这些改进通过以下方式提升了中文NLP解析： - 使常见日期格式如"2025年12"（无月）和"2025年12月30"（无日）能够正确解析 - 防止大小模式误匹配年份组件的错误 - 支持更多中文变体如"年份"后缀和不同分隔符 - 添加正确的文本片段检查以保持意图完整性 Log: 改进中文日期解析，支持更灵活的格式和更好的冲突解决 Influence: 1. 测试带或不带月/日后缀的各种中文日期格式 2. 验证混合查询如"2025-5的音乐"能正确解析为日期+文件类型 3. 测试边界情况如"50KB"与"2025年5月"的区别 4. 验证所有支持的日期分隔符（年./-）都能正常工作 Fixes: #370481
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @Johnson-zs, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Contributor
Author
|
/forcemerge |
1 similar comment
Contributor
Author
|
/forcemerge |
|
This pr force merged! (status: blocked) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
日 suffixes)
patterns
patterns
The changes improve Chinese NLP parsing by:
月30" (without 日) work
year components
separators
Log: Improved Chinese date parsing with more flexible formats and better
conflict resolution
Influence:
fix: 改进中文日期解析和处理功能
这些改进通过以下方式提升了中文NLP解析：
Log: 改进中文日期解析，支持更灵活的格式和更好的冲突解决
Influence:
Fixes: #370481