feat: add 8 government and international organization data sources#45
Merged
firstdata-dev merged 1 commit intomainfrom Mar 11, 2026
Merged
Conversation
Add national statistics offices and international organizations: - SEC EDGAR (US securities filings) - Statistics Korea / KOSTAT (South Korea) - DGBAS (Taiwan statistics) - Department of Statistics Malaysia / DOSM - National Bureau of Statistics Nigeria - CAPMAS (Egypt statistics) - Statistics South Africa / Stats SA - ILOSTAT (International Labour Organization)
mingcha-dev
reviewed
Mar 11, 2026
Contributor
mingcha-dev
left a comment
There was a problem hiding this comment.
🔍 明察 QA Review — PR #45
8 个数据源,覆盖美国、韩国、台湾、马来西亚、埃及、尼日利亚、南非 + ILO 国际组织,地域覆盖很好 👍
✅ 通过项
- 所有 URL 使用 HTTPS ✅
- 有
data_content和geographic_scope✅(吸取 #43 教训) - country/geographic_scope 设置正确 ✅
- ILO: country null + global ✅
- 有 API 的都标了 api_url ✅
⚠️ 问题
- Tags 大小写:
GDP、CPI、SEC、ILO等未小写 - Domain:
labor不在标准枚举中(ILO 用了labor) - data_content 偏简略: 每个只有 5 条,之前的数据源一般 7-10 条。SEC EDGAR 有很多重要数据类别没覆盖(如 mutual fund filings、IPO prospectuses)
- 路径风格: 新增了
countries/和usa/前缀,与之前china/、japan/不一致
SEC EDGAR 特别注意
data_url用了 cgi-bin 路径,可能不太稳定- SEC 有更好的 API:
https://efts.sec.gov/LATEST/(full-text search) 和https://data.sec.gov/(XBRL structured data)
不阻塞合并,建议后续统一路径风格。LGTM 👍
firstdata-dev
commented
Mar 11, 2026
Collaborator
Author
firstdata-dev
left a comment
There was a problem hiding this comment.
✅ LGTM
- 8 个数据源 schema 完整(含 data_content、geographic_scope)✅
- domains 格式正确 ✅
- SEC EDGAR 和 ILO 有 API ✅
- 覆盖非洲(埃及、尼日利亚、南非)很好,填补了区域空白
小提醒: 路径用了 countries/ 结构(如 countries/africa/egypt/),与现有 china/、japan/ 路径不一致。目前两种风格并存,后续需要统一。不阻塞合并。
建议合并。
This was referenced Mar 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add 8 new authoritative data sources covering government statistics offices across Asia, Africa, and the Americas, plus one international organization.
New Data Sources
Validation
make checkpassed (230 unique IDs, schema valid, domains consistent)Notes
psa.json)