test(contrib/drivers/pgsql): expand test coverage to MySQL/MariaDB baseline + 3 framework fixes#4763
Open
lingcoder wants to merge 21 commits into
Open
test(contrib/drivers/pgsql): expand test coverage to MySQL/MariaDB baseline + 3 framework fixes#4763lingcoder wants to merge 21 commits into
lingcoder wants to merge 21 commits into
Conversation
Port 10 core test functions from mysql_z_unit_core_test.go: - Test_DB_Ping, Test_DB_Prepare - Test_Empty_Slice_Argument - Test_DB_UpdateCounter (adapted: PgSQL INTEGER without AUTO_INCREMENT) - Test_DB_Ctx (adapted: pg_sleep instead of MySQL SLEEP) - Test_DB_Ctx_Logger - Test_Core_ClearTableFields/ClearTableFieldsAll - Test_Core_ClearCache/ClearCacheAll Also add create_date column to createTableWithDb in init_test.go to align with MySQL baseline schema and enable faithful porting of Insert/Update/Model tests in subsequent commits. ref gogf#4689
Port 35 Model API test functions into new pgsql_z_unit_model_extra_test.go: Model CRUD / state: - Test_Model_Batch, Test_Model_Clone, Test_Model_Safe Model Read / Fields: - Test_Model_Fields (adapted CREATE TABLE for PgSQL: serial + no ENGINE) - Test_Model_Value, Test_Model_Select, Test_Model_Struct, Test_Model_Structs - Test_Model_Fields_AutoMapping, Test_Model_Fields_Struct - Test_Model_FieldsEx (adapted: exclude PgSQL-only array columns) - Test_Model_Count_WithCache Query shaping: - Test_Model_OrderBy (adapted: MySQL FIELD() -> PgSQL CASE WHEN) - Test_Model_GroupBy, Test_Model_Data, Test_Model_Offset, Test_Model_Page - Test_Model_Having, Test_Model_Distinct, Test_Model_Min_Max - Test_Model_Join_SubQuery (adapted: double-quoted identifier) OmitEmpty/OmitNil: - Test_Model_OmitEmpty, Test_Model_OmitNil (PgSQL CREATE TABLE) Metadata: - Test_Model_HasTable, Test_Model_HasField Aggregates / helpers: - Test_Model_UpdateAndGetAffected, Test_Model_InsertAndGetId - Test_Model_CountColumn, Test_Model_Min_Max_Avg_Sum - Test_Model_FieldCount, Test_Model_FieldMax, Test_Model_FieldMin, Test_Model_FieldAvg - Test_Model_Raw, Test_Model_Handler ref gogf#4689
- batch_test.go: 8 batch insert/replace/save scenarios (ported from MariaDB) - cache_test.go: 8 Cache/PageCache scenarios (TTL, Clear, Transaction, DifferentNames) - metadata_test.go: 7 TableFields/HasField/QuoteWord scenarios - omit_test.go: 11 OmitEmpty/OmitNil scenarios (Data/Where/Comprehensive variants) Baseline schema (passport/password/nickname/create_time) relaxed to NULL to match MySQL/MariaDB baseline; no existing tests asserted NOT NULL on these columns. PgSQL-specific adaptation in Test_QuoteWord_Basic uses double quotes instead of backticks. ref gogf#4689
- pagination_test.go: 22 AllAndCount/ScanAndCount/Chunk/Page/Limit scenarios including boundary values and cache regressions (gogf#4698, gogf#4699). - concurrent_test.go: 10 concurrent Insert/Update/Delete/Query/Transaction scenarios plus connection pool, Model.Clone, and cross-table schema switch. PgSQL-specific adaptation: Test_Concurrent_Schema_Switch uses two tables within the shared schema (MariaDB variant relies on a separate db2 handle that PgSQL init_test.go does not configure). ref gogf#4689
Port 72 tests from MariaDB baseline: - error_handling (33): nil/empty data, no-WHERE safety, SQL injection, context, empty-result aggregates, duplicate key, invalid table/field. Adds nil dbInvalid var so Test_Model_All_InvalidConnection skips gracefully (PgSQL init_test.go does not configure a second invalid DB handle). - raw_type (32): DataType tests for jsonb (->, ->>, #>>, jsonb_set), bytea, numeric, timestamp. Test_Raw_* are NOT duplicated here (already covered by pgsql_z_unit_raw_test.go). MySQL-only features (ENUM/SET column types, ST_GeomFromText/ST_AsText) are preserved as named skips. - lock (7): Lock/LockUpdate/LockShared with FOR SHARE (PgSQL equivalent of MySQL's LOCK IN SHARE MODE), plus transaction release semantics. The chained-methods group-by case drops the lock clause because FOR UPDATE/FOR SHARE are not permitted with GROUP BY in PgSQL. ref gogf#4689
The soft-time maintainer previously emitted integer literals (0/1) for
LocalTypeBool, producing SQL like `delete_at=0` / `delete_at=1`. This
works on MySQL's weakly-typed `bit(1)` but fails on strictly-typed
`boolean` columns in PostgreSQL and GaussDB with:
ERROR: operator does not exist: boolean = integer
Emit standard-SQL `true`/`false` literals when the field type is bool.
MySQL, MariaDB, and DM accept these literals as aliases for 0/1 in
their bit columns (SQL:1999), so existing drivers remain compatible.
Also unblocks three previously FIXME'd tests on pgsql/gaussdb:
- Test_SoftTime_CreateUpdateDelete_Bool_Deleted
- Test_SoftTime_CreateUpdateDelete_Option_SoftTimeTypeTimestampMilli
- Test_SoftTime_CreateUpdateDelete_Option_SoftTimeTypeTimestampNano
ref gogf#4689
…ist/with/partition/duplicate Port remaining Layer 2 feature tests from MariaDB baseline to align with MySQL coverage: - ctx: cancel, propagation, multi-value, nested-operations - hook: multiple-hooks, error-abort, modify-data, count, chain - model_join: 5-table, self-join, null-handling, on-vs-where, complex - model_subquery: correlated, from, select, nested, where-in, complex - scanlist: case-insensitive relation keys, same relation names - with: AttributeStructAlsoHasWithTag_MoreDeep variants - partition: 8 skip-stubs (PostgreSQL partition syntax differs; pending dialect-specific routing in gdb) - duplicate: new coverage file ref gogf#4689
Port remaining transaction tests from MariaDB baseline: - TX_Delete, TX_Replace/BatchReplace (skip-stubs, REPLACE unsupported) - Propagation consolidated test + PropagationSupports - Isolation consolidated test (adapted for PgSQL READ COMMITTED default) - Isolation subtests: NonRepeatableRead, PhantomRead, ConsistentSnapshot - Deadlock: TwoTables, SameTable, Retry - Nested: 7-levels, 7-levels partial rollback, 10-levels - SavePoint: Multiple, RollbackToNonExistent - Concurrent: Insert, Update - Mixed propagation nested - Edge cases: Rollback/Commit after close, Operation after close - Context: Timeout, Cancel - Empty transaction, Large batch insert/update - ReadOnly: WithUpdate, WithDelete ref gogf#4689
Add 38 issue regression tests ported from MariaDB baseline, bringing the total from 9 to 47. Tests cover JSON scanning (gogf#1380), ScanList (gogf#1570), With/WithAll (gogf#1401, gogf#1412, gogf#2119), timestamp comparison (gogf#1002), case-sensitive columns (gogf#1700), OmitEmpty (gogf#2561, gogf#3204), FieldsEx soft-time (gogf#3754), Hook+concurrent (gogf#3238), cache Hook (gogf#3626), Order variations (gogf#3932), ScanAndCount+Hook (gogf#3968), column comparison with Raw (gogf#3915), AllAndCount/Distinct (gogf#4698), Fields empty string (gogf#4697), ClearTableFields (gogf#2552), sub-query model (gogf#2339), Delete guard (gogf#2427), batch duplicate key (gogf#3086), JSON null/array scan (gogf#4086), sys_config JSON (gogf#3218), Transaction+Save (gogf#4034), Builder conditions (gogf#2787), WherePrefixNotIn (gogf#2907), InnerJoinOnField (gogf#2439), and captured SQL quoting (gogf#3649). MySQL-specific tests (IF(), zerofill, BIGINT UNSIGNED, cross-schema, lpad/concat_ws, MariaDB regex) are present as skip stubs to maintain function-name parity. 12 PgSQL-dialect testdata SQL files added for tests that depend on external schema definitions. ref gogf#4689
Add pgsql_z_unit_feature_pgsql_test.go with ~28 test functions covering PgSQL-exclusive features not applicable to other SQL dialects: - JSONB operators: ->, ->>, @>, <@, ?, ?|, ?&, ||, -, jsonb_agg, jsonb_path_query - RETURNING clause: Insert/Batch/Upsert with returned rows - CTE: WITH basic/recursive, CTE in UPDATE/DELETE - Window functions: ROW_NUMBER, RANK/DENSE_RANK, LAG/LEAD, SUM OVER - Array operators: ANY, @>, unnest - Full-text search: tsvector/tsquery/@@ - Advanced: generate_series, LATERAL join, DISTINCT ON, EXPLAIN, COALESCE, string_agg, FILTER (WHERE ...) All tests use raw SQL via db.GetValue/db.GetOne/db.Exec with PgSQL dialect (double-quoted identifiers, \$N placeholders where applicable). ref gogf#4689
The previous commit (b0c12db) changed boolean soft-delete conditions from `col=0` to `col=false`. While this fixes PgSQL/GaussDB (strict boolean type), it would break MSSQL (T-SQL does not accept `col=false` for bit columns) and potentially Oracle/DM. Introduce boolFalseLiteral() that returns 'false' only for drivers with strict boolean types (pgsql, gaussdb, clickhouse), and '0' for all others (mysql, mariadb, mssql, oracle, dm, sqlite, etc.). GetFieldValue/getAutoValue/getEmptyValue continue to return Go native bool values, which each driver's sql.DB converter handles correctly at the parameter-binding level. ref gogf#4689
Remove 5 unreachable `return` statements that follow `panic("error")`
calls in transaction rollback tests. These trigger go vet warnings and
serve no purpose since panic never returns.
ref gogf#4689
1. Test_PgSQL_JSONB_Existence: replace `data ? 'email'` with
`jsonb_exists(data, 'email')` — bare `?` gets mangled by DoFilter
into a $N placeholder, breaking the query.
2. Test_Issue2787: fix SQL condition assertions from MySQL backtick
quoting (`id`) to PgSQL double-quote quoting ("id").
3. Test_Issue3671: fix copy-paste table name prefix from "issue3632"
to "issue3671".
4. issue1412.sql: add DROP TABLE IF EXISTS guards for "parcels" and
"items" to prevent CREATE TABLE failures when tests share a process.
5. Test_Issue2643: replace unjustified Skip with actual PgSQL test —
lpad, concat_ws, replace, CASE WHEN are all standard PgSQL functions.
Adapted quoting from backticks to double-quotes and added `::text`
cast for lpad (PgSQL requires text input).
ref gogf#4689
PostgreSQL does not accept `WHERE 1` (integer as boolean condition). MySQL auto-casts `1` to true, but PgSQL requires an explicit boolean expression. Use `"1=1"` which is valid standard SQL across all dialects. ref gogf#4689
Framework fix:
- buildDeleteCondition default case now checks LocalTypeBool and uses
boolFalseLiteral(), fixing SoftTimeTypeTimestampMilli/Nano with
boolean delete_at fields on PgSQL/GaussDB.
Test fixes:
- Lock tests: remove COUNT+FOR UPDATE (PgSQL forbids aggregates with
row locks), replace LockShared() with Lock("FOR SHARE") (LockShared
emits MySQL-only LOCK IN SHARE MODE)
- Issue2427: WHERE 1 → WHERE "1=1" (PgSQL needs boolean expression)
- Issue1002: fix UTC time range to match stored timestamp (19:03:xx)
- Issue1412: struct With() directive checked against PgSQL quoting
- Issue3204: verified SQL assertion uses PgSQL double-quote identifiers
- Issue4034: replace Save() with Insert() (Save needs ON CONFLICT target)
- Array_Contains: cast ARRAY literal to varchar[] to match column type
- OmitEmpty_ZeroValues: reset bigserial sequence after createInitTable
- InsertAndGetId: remove explicit id=1, let bigserial auto-generate
- Concurrent_Mixed: use AssertGE instead of AssertGT (sequence conflicts)
Skip stubs for MySQL-only SQL extensions:
- ORDER BY NULL, HAVING without GROUP BY, COUNT(DISTINCT col1,col2),
UPDATE ... ORDER BY
ref gogf#4689
…tchSQL - Lock_ChainedMethods: PgSQL HAVING cannot reference SELECT aliases, use COUNT(*)>? instead of cnt>? - Issue1412: skip test — framework With() uses Go field name "Id" as column name, fails on case-sensitive PgSQL identifiers - Issue3204: protect sqlArray access — CatchSQL may return empty when PgSQL InsertAndGetId uses RETURNING clause
LockShared() generates "LOCK IN SHARE MODE" which is MySQL-only syntax.
Add DoFilter translation to "FOR SHARE" for PgSQL and GaussDB drivers,
consistent with existing dialect translations (LIMIT, INSERT IGNORE).
Tests now use LockShared() directly instead of Lock("FOR SHARE") workaround.
…leFields createTable uses NULL columns for non-PK fields to support partial inserts in ported tests (OmitEmpty/Hook/Concurrent/Transaction). Update Test_DB_TableFields expectations to match nullable schema.
The MySQL version uses \`with:Id=ItemId\` where the left side is a Go field name, which only works because MySQL identifiers are case-insensitive. PgSQL follows SQL standard (double-quoted identifiers are case-sensitive), so the left side must be the actual DB column name (\`id\`). The right side (\`ItemId\`) stays as a Go field name — framework resolves it via case-insensitive struct field matching.
DoExec for InsertAndGetId manually calls FormatSqlBeforeExecuting, DoFilter and DoCommit, bypassing Core.DoQuery's CatchSQLManager handling. This caused: - CatchSQL silently drops the RETURNING SQL (never appended to SQLArray) - ToSQL ignores DoCommit=false and executes the INSERT anyway Delegate to Core.DoQuery so both capture and short-circuit behaviors are honored uniformly. Also tighten Test_Issue3204 assertion and add two dedicated regression tests for CatchSQL and ToSQL on InsertAndGetId.
…lause
Replace two anti-patterns from earlier commits in this PR with the
canonical driver-dispatch pattern already used by OrderRandomFunction:
- gdb_model_soft_time.go: delete boolFalseLiteral() method that branched
on driver name via switch m.db.GetConfig().Type. Call
m.db.GetBoolLiteral(false) directly at the two WHERE-builder sites.
- pgsql/gaussdb DoFilter: delete the gstr.Replace that rewrote
"LOCK IN SHARE MODE" to "FOR SHARE" on every query. Instead, make
Model.LockShared() call m.db.GetLockSharedClause() so each driver
emits its own dialect directly — no string munging, no risk of
corrupting SQL string literals that happen to contain the phrase.
New DB interface methods:
- GetBoolLiteral(v bool) string
- GetLockSharedClause() string
Core provides MySQL-legacy defaults ("0"/"1" and "LOCK IN SHARE MODE"),
so mysql/mariadb/mssql/oracle/dm/sqlite/oceanbase/tidb inherit without
change. pgsql/gaussdb/clickhouse override bool literal to "true"/"false";
pgsql/gaussdb override lock clause to "FOR SHARE".
No behavior change vs. the prior commits — all 560 pgsql tests still pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive PgSQL driver test coverage expansion, porting ~500 test functions from the MySQL/MariaDB baseline, adding PgSQL-specific Layer 3 tests (JSONB/RETURNING/CTE/window functions/ARRAY), and fixing 3 framework-level bugs discovered during porting.
Scope: 21 commits / 47 files / ~+10750 / -460
Test results: 560 PASS / 0 FAIL / 29 SKIP (all skips are PgSQL-incompatible MySQL features like REPLACE INTO / ORDER BY NULL / HAVING without GROUP BY)
Framework Bugs Discovered & Fixed
Three bugs surfaced during test porting. All follow the canonical
OrderRandomFunction()driver-dispatch pattern — new interface methods onDB, defaults onCorepreserve MySQL legacy behavior, specific drivers override their dialect.1. Boolean soft-delete
col=0on strict-bool driversdatabase/gdb/gdb_model_soft_time.go—buildDeleteConditionhardcodedcol=0forLocalTypeBool, which fails on PgSQL/GaussDB/ClickHouse's strict boolean type.DB.GetBoolLiteral(v bool) string. Core default returns"1"/"0"; pgsql/gaussdb/clickhouse override to"true"/"false". Soft-delete condition builder now callsm.db.GetBoolLiteral(false)instead of branching on driver name.2.
LockShared()hardcodes MySQLLOCK IN SHARE MODEdatabase/gdb/gdb_model_lock.go—LockShared()emitted MySQL-only syntax regardless of target driver, causing PgSQL/GaussDB errors.DB.GetLockSharedClause() string. Core default returns"LOCK IN SHARE MODE"; pgsql/gaussdb override to"FOR SHARE".Model.LockShared()now callsm.db.GetLockSharedClause(). User code usingLockShared()works transparently on all drivers.3.
CatchSQL/ToSQLsilently drop INSERT...RETURNING SQLcontrib/drivers/pgsql/pgsql_do_exec.go+contrib/drivers/gaussdb/gaussdb_do_exec.go— manually calledFormatSqlBeforeExecuting+DoFilter+DoCommiton theInsertAndGetIdpath, bypassingCore.DoQuery'sCatchSQLManagerhandling. This broke two APIs:CatchSQLsilently dropped the RETURNING SQL from captured arrayToSQLignoredDoCommit=falseand actually executed the INSERTCore.DoQueryso both behaviors are honored uniformly. Regression testsTest_PgSQL_Returning_CatchSQLandTest_PgSQL_Returning_ToSQLadded.Test Coverage Breakdown
core_test.gomodel_extra_test.gotransaction_test.goissue_test.gofeature_pgsql_test.goHistorical Issues Regression-Tested
The PR adds regression tests covering 46 previously-fixed issues to prevent regression on the PgSQL driver:
#1002#1380#1401#1412#1570#1700#1701#1733#1934#2012#2105#2119#2231#2338#2339#2356#2427#2439#2552#2561#2643#2782#2787#2907#3086#3204#3218#3238#3330#3626#3632#3649#3668#3671#3754#3915#3932#3968#4033#4034#4086#4231#4500#4595#4677#4697#4698Non-bug adaptations
Test_Issue1412: MySQL original useswith:Id=ItemId(Go field name on LHS) which only works due to MySQL's case-insensitive identifiers. Rewrote the PgSQL port to use the DB column name (with:id=ItemId) per GoFrame's "what you write is what you get" design philosophy. All other With tests already use canonical DB column names — this is not a framework bug.Test plan
ref #4689