Skip to content

Commit 45317db

Browse files
Copilotdr5hn
Copilot
and
dr5hn
authored
Fix schema file formats: Add PostgreSQL schema export and remove SQL Server schema (#1332)
* Initial plan * docs: initial investigation of psql/schema.sql issue Co-authored-by: dr5hn <6929121+dr5hn@users.noreply.github.com> * fix: remove incorrect MySQL-formatted psql/schema.sql file Co-authored-by: dr5hn <6929121+dr5hn@users.noreply.github.com> * docs: add fix documentation for psql/schema.sql removal Co-authored-by: dr5hn <6929121+dr5hn@users.noreply.github.com> * fix: add PostgreSQL schema export and remove SQL Server schema file Co-authored-by: dr5hn <6929121+dr5hn@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: dr5hn <6929121+dr5hn@users.noreply.github.com>
1 parent dc0f796 commit 45317db

File tree

8 files changed

+178
-377
lines changed

8 files changed

+178
-377
lines changed

‎.github/fixes-docs/FIX_SCHEMA_FILES_FORMAT.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Fix Summary: Schema Files Format Issue
2+
3+
## Issue Reference
4+
**Original Issue:** [Bug]: psql/schema.sql is actually MySQL
5+
**Issue Type:** API/Export Issue
6+
7+
## Executive Summary
8+
9+
The file `psql/schema.sql` was incorrectly generated using MySQL's `mysqldump` command instead of PostgreSQL format. Similarly, `sqlserver/schema.sql` was also using MySQL format. This fix adds proper PostgreSQL schema export using `pg_dump --schema-only` and removes the unnecessary SQL Server schema file.
10+
11+
---
12+
13+
## Problem Statement
14+
15+
### What was wrong?
16+
1. `psql/schema.sql` was initially generated using `mysqldump` (MySQL format)
17+
2. `sqlserver/schema.sql` was also generated using `mysqldump` (MySQL format)
18+
3. File headers showed MySQL-specific syntax:
19+
```sql
20+
-- MySQL dump 10.13 Distrib 8.0.44, for Linux (x86_64)
21+
```
22+
23+
### Why was this a problem?
24+
- Users downloading `psql/schema.sql` expecting PostgreSQL format would encounter incompatibilities
25+
- MySQL-specific syntax like `AUTO_INCREMENT`, backtick identifiers, and MySQL comments are not compatible with PostgreSQL
26+
- SQL Server also has different syntax requirements
27+
28+
---
29+
30+
## Changes Made (Final Solution)
31+
32+
### 1. Added proper PostgreSQL schema export
33+
**File:** `.github/workflows/export.yml`
34+
**Change:** Added `pg_dump --schema-only` command to generate proper PostgreSQL schema
35+
36+
```yaml
37+
- name: Export PostgreSQL SQL
38+
env:
39+
PGPASSWORD: postgres
40+
run: |
41+
mkdir -p psql
42+
# Export PostgreSQL schema only (no data)
43+
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --schema-only --clean --if-exists --no-owner --no-acl > psql/schema.sql
44+
# Export individual tables with data
45+
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --inserts --clean --if-exists --no-owner --no-acl -t regions > psql/regions.sql
46+
# ... (other tables)
47+
```
48+
49+
### 2. Removed SQL Server schema generation
50+
**File:** `.github/workflows/export.yml`
51+
**Change:** Removed line that generated MySQL dump as `sqlserver/schema.sql`
52+
53+
```diff
54+
- name: Generate Schema Files
55+
run: |
56+
echo "📋 Generating schema files..."
57+
# Export MySQL schema only (no data)
58+
mysqldump -uroot -proot --no-data --single-transaction --add-drop-table world > sql/schema.sql
59+
- # Also export for other formats
60+
- mysqldump -uroot -proot --no-data --single-transaction --add-drop-table world > sqlserver/schema.sql
61+
echo "✅ Schema files generated"
62+
```
63+
64+
### 3. Deleted SQL Server schema file
65+
**File:** `sqlserver/schema.sql`
66+
**Action:** Removed from repository (MySQL format was incorrect for SQL Server)
67+
68+
### 4. Updated .gitignore
69+
**File:** `.gitignore`
70+
**Change:** Updated to track `psql/schema.sql` and exclude `sqlserver/schema.sql`
71+
72+
```diff
73+
# Keep schema files - they are small and useful
74+
!sql/schema.sql
75+
-!sqlserver/schema.sql
76+
-# Note: psql/schema.sql excluded - PostgreSQL schema is properly exported via pg_dump commands
77+
+!psql/schema.sql
78+
+# Note: sqlserver/schema.sql excluded - not needed for SQL Server exports
79+
```
80+
81+
---
82+
83+
## Rationale
84+
85+
### PostgreSQL Schema Export
86+
Using `pg_dump --schema-only` provides:
87+
- Proper PostgreSQL-compatible SQL syntax
88+
- CREATE TABLE statements with PostgreSQL data types
89+
- Correct constraints and foreign keys for PostgreSQL
90+
- No data, only schema definitions
91+
92+
### SQL Server
93+
SQL Server exports already include schema in the individual table exports. A separate MySQL-formatted schema file would be incompatible and misleading.
94+
95+
---
96+
97+
## Directory Structure (After Fix)
98+
99+
### psql/ (PostgreSQL)
100+
```
101+
psql/
102+
├── schema.sql (PostgreSQL format, schema only - NEW)
103+
├── regions.sql (PostgreSQL format, with data)
104+
├── subregions.sql (PostgreSQL format, with data)
105+
├── countries.sql (PostgreSQL format, with data)
106+
├── states.sql (PostgreSQL format, with data)
107+
├── cities.sql.gz (PostgreSQL format, compressed)
108+
└── world.sql.gz (PostgreSQL format, complete DB)
109+
```
110+
111+
### sqlserver/ (SQL Server)
112+
```
113+
sqlserver/
114+
├── regions.sql (SQL Server format, with data)
115+
├── subregions.sql (SQL Server format, with data)
116+
├── countries.sql (SQL Server format, with data)
117+
├── states.sql (SQL Server format, with data)
118+
├── cities.sql.gz (SQL Server format, compressed)
119+
└── world.sql.gz (SQL Server format, complete DB)
120+
```
121+
**Note:** No `schema.sql` file - schema is included in the individual table exports.
122+
123+
### sql/ (MySQL)
124+
```
125+
sql/
126+
├── schema.sql (MySQL format, schema only)
127+
├── regions.sql (MySQL format, with data)
128+
└── ... (other files)
129+
```
130+
131+
---
132+
133+
## Validation
134+
135+
### Files Checked
136+
✅ YAML syntax validation passed for export.yml
137+
✅ `.gitignore` properly configured
138+
✅ PostgreSQL schema export command uses correct flags
139+
✅ SQL Server MySQL-formatted schema removed
140+
141+
### Command Flags Explanation
142+
- `--schema-only`: Export only schema (CREATE statements), no data
143+
- `-Fp`: Plain text format
144+
- `--clean`: Include DROP statements before CREATE
145+
- `--if-exists`: Use IF EXISTS with DROP statements
146+
- `--no-owner`: Don't include ownership commands
147+
- `--no-acl`: Don't include access privileges
148+
149+
---
150+
151+
## Testing Recommendations
152+
153+
When the workflow runs:
154+
1. Verify `psql/schema.sql` is generated with PostgreSQL syntax
155+
2. Verify `sqlserver/schema.sql` is NOT generated
156+
3. Confirm schema file can be imported into PostgreSQL: `psql -U postgres -d testdb -f psql/schema.sql`
157+
4. Check that CREATE TABLE statements use PostgreSQL data types (e.g., `integer`, `character varying`, `timestamp`)
158+
159+
---
160+
161+
## Data Sources & References
162+
163+
- GitHub Issue: [Bug]: psql/schema.sql is actually MySQL
164+
- Maintainer feedback: Add proper PostgreSQL schema export, remove SQL Server schema
165+
- PostgreSQL pg_dump documentation: https://www.postgresql.org/docs/current/app-pgdump.html
166+
- MySQL mysqldump documentation: https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html
167+
168+
---
169+
170+
**Fix completed:** January 11, 2026
171+
**Updated based on maintainer feedback:** Added proper PostgreSQL schema export with `pg_dump --schema-only`, removed SQL Server schema file

‎.github/workflows/export.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -224,16 +224,16 @@ jobs:
224224
echo "📋 Generating schema files..."
225225
# Export MySQL schema only (no data)
226226
mysqldump -uroot -proot --no-data --single-transaction --add-drop-table world > sql/schema.sql
227-
# Also export for other formats
228-
mysqldump -uroot -proot --no-data --single-transaction --add-drop-table world > psql/schema.sql
229-
mysqldump -uroot -proot --no-data --single-transaction --add-drop-table world > sqlserver/schema.sql
230227
echo "✅ Schema files generated"
231228
232229
- name: Export PostgreSQL SQL
233230
env:
234231
PGPASSWORD: postgres
235232
run: |
236233
mkdir -p psql
234+
# Export PostgreSQL schema only (no data)
235+
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --schema-only --clean --if-exists --no-owner --no-acl > psql/schema.sql
236+
# Export individual tables with data
237237
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --inserts --clean --if-exists --no-owner --no-acl -t regions > psql/regions.sql
238238
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --inserts --clean --if-exists --no-owner --no-acl -t subregions > psql/subregions.sql
239239
pg_dump --dbname=postgresql://postgres:postgres@localhost/world -Fp --inserts --clean --if-exists --no-owner --no-acl -t countries > psql/countries.sql

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ csv/translations.csv
5252
# Keep schema files - they are small and useful
5353
!sql/schema.sql
5454
!psql/schema.sql
55-
!sqlserver/schema.sql
55+
# Note: sqlserver/schema.sql excluded - not needed for SQL Server exports
5656

5757
# Always track ALL compressed files (.gz)
5858
!*.sql.gz

‎bin/db/schema.sql‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,6 @@ CREATE TABLE `cities` (
198198
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
199199
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
200200

201-
-- Dump completed on 2025-12-13 9:08:52
201+
-- Dump completed on 2026-01-11 19:44:09
202202

203203
SET FOREIGN_KEY_CHECKS=1;

‎contributions/cities/BQ.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -327,4 +327,4 @@
327327
"flag": 1,
328328
"wikiDataId": "Q2511504"
329329
}
330-
]
330+
]

‎contributions/countries/countries.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16747,4 +16747,4 @@
1674716747
"flag": 1,
1674816748
"wikiDataId": "Q26273"
1674916749
}
16750-
]
16750+
]

0 commit comments

Comments
 (0)