Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Imports a MySQL dump file into the SQLite database. Parses SQL statements using
**Parameters:**

- `<file>` (required), Path to the SQL dump file. Use `-` to read from STDIN.
- `[--enable-ast-driver]` Use the new AST-based `WP_SQLite_Driver` instead of the legacy `WP_SQLite_Translator` for full MySQL compatibility.
- `[--enable-ast-driver]` Enable the AST driver on older integration plugin releases. The current `WP_MySQL_On_SQLite` API is selected automatically when available.

### `wp sqlite export [<file>]`

Expand All @@ -23,7 +23,7 @@ Exports the SQLite database to a MySQL-compatible SQL dump file. Generates `DROP
- `[--tables=<tables>]` Comma-separated list of tables to include. If omitted, all tables are exported.
- `[--exclude_tables=<tables>]` Comma-separated list of tables to skip.
- `[--porcelain]` Output only the filename (for scripting).
- `[--enable-ast-driver]` Use the new AST-based driver.
- `[--enable-ast-driver]` Enable the AST driver on older integration plugin releases.

### `wp sqlite tables`

Expand All @@ -32,13 +32,13 @@ Lists all user tables in the SQLite database, excluding internal/system tables.
**Parameters:**

- `[--format=<format>]` Output format: `list` (default, one table per line), `csv`, or `json`.
- `[--enable-ast-driver]` Use the new AST-based driver.
- `[--enable-ast-driver]` Enable the AST driver on older integration plugin releases.

## Tech Stack

- PHP >=7.4, WP-CLI package (`wp-cli/wp-cli ^2.5`)
- WordPress SQLite Database Integration plugin, translation layer between MySQL queries and SQLite
- Two driver modes: legacy `WP_SQLite_Translator` and new AST-based `WP_SQLite_Driver` (via `--enable-ast-driver`)
- The PDO-based `WP_MySQL_On_SQLite` API is selected automatically when available; `WP_SQLite_Translator` and `WP_SQLite_Driver` remain as compatibility fallbacks for older integration plugin releases

## Directory Structure

Expand Down Expand Up @@ -105,13 +105,13 @@ Runs lint, phpcs, phpunit, and behat sequentially.
- `command.php` is the entry point, it registers `wp sqlite` as a WP-CLI command pointing to `SQLite_Command`
- `SQLite_Command` delegates to `Import`, `Export`, and `Tables` classes
- `SQLiteDatabaseIntegrationLoader::load_plugin()` bootstraps the WordPress SQLite Database Integration plugin, which MUST be loaded before any database operations
- `SQLiteDriverFactory::create_driver()` creates the appropriate driver based on whether the AST driver is enabled
- `SQLiteDriverFactory::create_driver()` creates the PDO-based `WP_MySQL_On_SQLite` driver when the class is available, with compatibility fallbacks for older integration plugin versions and the legacy translator
- The import parser (`Import::parse_statements()`) is a streaming SQL parser using PHP generators, it handles quotes, comments, escape sequences, and multi-line statements character by character

## Common Pitfalls

- **MUST load the SQLite plugin first:** All database operations require `SQLiteDatabaseIntegrationLoader::load_plugin()` to be called before creating a driver. The driver depends on classes from the WordPress SQLite Database Integration plugin.
- **Two driver APIs:** The legacy `WP_SQLite_Translator` and new `WP_SQLite_Driver` (AST) have different class names. Use `SQLiteDriverFactory::create_driver()` to get the correct one, do not instantiate drivers directly.
- **Multiple driver APIs:** Always use `SQLiteDriverFactory::create_driver()`. It selects the PDO-based `WP_MySQL_On_SQLite` API when available and preserves compatibility with older AST drivers and the legacy translator.
- **SQL parser edge cases:** The `Import::parse_statements()` method handles escape sequences, nested quotes, and multi-line comments manually. Changes to this parser MUST be tested with the existing Behat scenarios in `features/sqlite-import.feature`.
- **Encoding handling:** Import attempts UTF-8 conversion as a fallback when queries fail. This is intentional error recovery, not a bug.
- **`@when after_wp_config_load`:** All command methods use this WP-CLI hook annotation. New commands MUST include it to ensure WordPress config is loaded before execution.
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ $ wp sqlite import <file>
The path to the MySQL compatible dump file to import. When passing `-` as the file argument, the SQL commands are read from standard input.

[--enable-ast-driver]
Enables new AST driver for full MySQL compatibility.
Enables the AST driver for older integration plugin releases. The current driver API is selected automatically.

### wp sqlite export

Expand All @@ -46,7 +46,7 @@ $ wp sqlite export [<file>] [--tables=<tables>] [--exclude-tables] [--porcelain]
Output filename for the exported database.

[--enable-ast-driver]
Enables new AST driver for full MySQL compatibility.
Enables the AST driver for older integration plugin releases. The current driver API is selected automatically.

### wp sqlite tables

Expand All @@ -68,7 +68,7 @@ $ wp sqlite tables [--format=<list|csv>]
---

[--enable-ast-driver]
Enables new AST driver for full MySQL compatibility.
Enables the AST driver for older integration plugin releases. The current driver API is selected automatically.

**EXAMPLES**

Expand Down
77 changes: 62 additions & 15 deletions features/bootstrap/SQLiteFeatureContext.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
class SQLiteFeatureContext extends WPCLIFeatureContext implements Context {

private $db;
private $comment_injection_table_name;

/**
* @Given /^a SQL dump file named "([^"]*)" with content:$/
Expand Down Expand Up @@ -107,8 +108,13 @@ public function theSqliteDatabaseContainsSomeSampleData() {
$this->connectToDatabase();
$this->db->exec(
"
INSERT OR REPLACE INTO wp_posts (ID, post_title, post_content, post_type, post_status)
VALUES (1, 'Sample Post', 'This is a sample post content.', 'post', 'publish');
INSERT OR REPLACE INTO wp_posts (
ID, post_title, post_content, post_excerpt, to_ping, pinged,
post_content_filtered, post_type, post_status
)
VALUES (
1, 'Sample Post', 'This is a sample post content.', '', '', '', '', 'post', 'publish'
);
"
);

Expand Down Expand Up @@ -274,25 +280,66 @@ public function theSqlStringEscapeBytesShouldBePreserved() {
* @Given /^the SQLite database contains a test table with alphanumeric string hash values$/
*/
public function theSqliteDatabaseContainsATestTableWithAlphanumericStringHashValues() {
$this->connectToDatabase();

// Create a test table with hash values that look like scientific notation
$this->db->exec( 'DROP TABLE IF EXISTS test_export_alphanumeric_string' );
$this->db->exec(
'
$this->create_file(
'test_export_alphanumeric_string_setup.sql',
"
DROP TABLE IF EXISTS test_export_alphanumeric_string;
CREATE TABLE test_export_alphanumeric_string (
id INTEGER PRIMARY KEY,
hash_value TEXT
)
'
);
INSERT INTO test_export_alphanumeric_string (id, hash_value) VALUES (1, '123e99');
"
);
$this->wpcli_tests_invoke_proc(
$this->proc( 'wp sqlite import test_export_alphanumeric_string_setup.sql' ),
'run'
);
}

// Insert test data with values that might be mistaken for scientific notation
$this->db->exec(
/**
* @Given /^the SQLite database contains a table with a backtick in its name$/
*/
public function theSqliteDatabaseContainsATableWithABacktickInItsName() {
$this->create_file(
'test_export_identifier_setup.sql',
"
INSERT INTO test_export_alphanumeric_string (id, hash_value) VALUES
(1, '123e99')
"
DROP TABLE IF EXISTS `test``table`;
CREATE TABLE `test``table` (id INTEGER PRIMARY KEY, value TEXT);
INSERT INTO `test``table` (id, value) VALUES (1, 'Test value');
"
);
$this->wpcli_tests_invoke_proc(
$this->proc( 'wp sqlite import test_export_identifier_setup.sql' ),
'run'
);
}

/**
* @Given /^the SQLite database contains a table with a comment injection name$/
*/
public function theSqliteDatabaseContainsATableWithACommentInjectionName() {
$this->connectToDatabase();

$this->comment_injection_table_name = "safe`\nDROP TABLE IF EXISTS wp_users; --\rtable";
$quoted_table_name = '"' . str_replace( '"', '""', $this->comment_injection_table_name ) . '"';

$this->db->exec( "CREATE TABLE $quoted_table_name (id INTEGER PRIMARY KEY, value TEXT)" );
$this->db->exec( "INSERT INTO $quoted_table_name (id, value) VALUES (1, 'Test value')" );
}

/**
* @When /^I export the table with a comment injection name$/
*/
public function iExportTheTableWithACommentInjectionName() {
$this->wpcli_tests_invoke_proc(
$this->proc(
sprintf(
'wp sqlite --enable-ast-driver export test_export_comment.sql --tables=%s',
escapeshellarg( $this->comment_injection_table_name )
)
),
'run'
);
}
}
24 changes: 24 additions & 0 deletions features/sqlite-export.feature
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,30 @@ Feature: WP-CLI SQLite Export Command
INSERT INTO `test_export_alphanumeric_string` VALUES (1,123e99);
"""

@require-sqlite
Scenario: Export should quote table names containing backticks
Given the SQLite database contains a table with a backtick in its name
When I run `wp sqlite --enable-ast-driver export test_export_identifier.sql`
Then the file "test_export_identifier.sql" should contain:
"""
DROP TABLE IF EXISTS `test``table`;
"""
And the file "test_export_identifier.sql" should contain:
"""
INSERT INTO `test``table` VALUES (1,'Test value');
"""

@require-sqlite
Scenario: Export should keep multiline table names inside comments
Given the SQLite database contains a table with a comment injection name
When I export the table with a comment injection name
And I run `wp sqlite --enable-ast-driver import test_export_comment.sql`
Then STDOUT should contain:
"""
Success: Imported from 'test_export_comment.sql'.
"""
And the SQLite database should contain a table named "wp_users"

@require-sqlite
Scenario: Export should preserve serialized settings containing a NUL byte
Given the SQLite database contains a serialized settings option with a NUL byte separator
Expand Down
5 changes: 5 additions & 0 deletions phpstan/stubs.php
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ public function query( string $sql ) {}
public function get_connection(): WP_SQLite_Connection {}
}

// phpcs:ignore WordPress.DB.RestrictedClasses.mysql__PDO -- Stub for the plugin's PDO subclass.
class WP_MySQL_On_SQLite extends PDO {
public function get_connection(): WP_SQLite_Connection {}
}

class WP_SQLite_Translator {
public function __construct() {}
public function query( string $sql ) {}
Expand Down
78 changes: 53 additions & 25 deletions src/Export.php
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@
use Exception;
use PDO;
use WP_CLI;
use WP_MySQL_On_SQLite;
use WP_SQLite_Driver;
use WP_SQLite_Translator;

class Export {
/**
* The SQLite driver instance.
*
* @var WP_SQLite_Driver|WP_SQLite_Translator
* @var WP_MySQL_On_SQLite|WP_SQLite_Driver|WP_SQLite_Translator
*/
protected $driver;

Expand Down Expand Up @@ -85,8 +86,7 @@ protected function close_output_stream( $handle ) {
protected function write_sql_statements( $handle ) {
$include_tables = $this->get_include_tables();
$exclude_tables = $this->get_exclude_tables();
foreach ( $this->driver->query( 'SHOW TABLES' ) as $row ) {
$table_name = array_values( (array) $row )[0];
foreach ( $this->get_table_names() as $table_name ) {

// Skip tables that are not in the include_tables list if the list is defined
if ( ! empty( $include_tables ) && ! in_array( $table_name, $include_tables, true ) ) {
Expand All @@ -111,6 +111,24 @@ protected function write_sql_statements( $handle ) {
fwrite( $handle, sprintf( '-- Dump completed on %s', gmdate( 'c' ) ) );
}

/**
* Get the database table names.
*
* @return string[]
*/
protected function get_table_names() {
$result = $this->driver->query( 'SHOW TABLES' );
if ( $this->driver instanceof PDO ) {
return $result->fetchAll( PDO::FETCH_COLUMN );
}

$tables = array();
foreach ( $result as $row ) {
$tables[] = array_values( (array) $row )[0];
}
return $tables;
}

/**
* Write the create statement for a table to the output stream.
*
Expand All @@ -120,9 +138,10 @@ protected function write_sql_statements( $handle ) {
* @throws Exception
*/
protected function write_create_table_statement( $handle, $table_name ) {
$comment = $this->get_dump_comment( sprintf( 'Table structure for table `%s`', $table_name ) );
$quoted_table_name = $this->quote_identifier( $table_name );
$comment = $this->get_dump_comment( sprintf( 'Table structure for table %s', $quoted_table_name ) );
fwrite( $handle, $comment . PHP_EOL . PHP_EOL );
fwrite( $handle, sprintf( 'DROP TABLE IF EXISTS `%s`;', $table_name ) . PHP_EOL );
fwrite( $handle, sprintf( 'DROP TABLE IF EXISTS %s;', $quoted_table_name ) . PHP_EOL );
fwrite( $handle, $this->get_create_statement( $table_name ) . PHP_EOL );
}

Expand All @@ -140,7 +159,7 @@ protected function write_insert_statements( $handle, $table_name ) {
return;
}

$comment = $this->get_dump_comment( sprintf( 'Dumping data for table `%s`', $table_name ) );
$comment = $this->get_dump_comment( sprintf( 'Dumping data for table %s', $this->quote_identifier( $table_name ) ) );
fwrite( $handle, $comment . PHP_EOL . PHP_EOL );
foreach ( $this->get_insert_statements( $table_name ) as $insert_statement ) {
fwrite( $handle, $insert_statement . PHP_EOL );
Expand All @@ -158,13 +177,14 @@ protected function write_insert_statements( $handle, $table_name ) {
* @throws Exception
*/
protected function get_create_statement( $table_name ) {
$table_name = $this->driver instanceof WP_SQLite_Driver
? $this->driver->get_connection()->quote_identifier( $table_name )
: $table_name;
$create = $this->driver->query( 'SHOW CREATE TABLE ' . $this->quote_identifier( $table_name ) );
if ( $this->driver instanceof PDO ) {
$sql = $create->fetchColumn( 1 );
return rtrim( $sql, ';' ) . ";\n";
}

$create = $this->driver->query( 'SHOW CREATE TABLE ' . $table_name );
$sql = $create[0]->{'Create Table'};
$sql = rtrim( $sql, ';' ); // The old SQLite driver appends a semicolon.
$sql = $create[0]->{'Create Table'};
$sql = rtrim( $sql, ';' ); // The old SQLite driver appends a semicolon.
return $sql . ";\n";
}

Expand All @@ -176,12 +196,11 @@ protected function get_create_statement( $table_name ) {
* @return \Generator
*/
protected function get_insert_statements( $table_name ) {
$pdo = $this->get_pdo();
$stmt = $pdo->prepare( 'SELECT * FROM ' . $table_name );
$stmt->execute();
$quoted_table_name = $this->quote_identifier( $table_name );
$stmt = $this->get_sqlite_pdo()->query( 'SELECT * FROM ' . $quoted_table_name );
// phpcs:ignore
while ( $row = $stmt->fetch( PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT ) ) {
yield sprintf( 'INSERT INTO `%1s` VALUES (%2s);', $table_name, $this->escape_values( $pdo, $row ) );
yield sprintf( 'INSERT INTO %1s VALUES (%2s);', $quoted_table_name, $this->escape_values( $row ) );
}
}

Expand Down Expand Up @@ -228,13 +247,11 @@ protected function get_include_tables() {
/**
* Escape values for insert statement
*
* @param PDO $pdo
* @param $values
*
* @return string
*/
protected function escape_values( PDO $pdo, $values ) {
// Get a mysql PDO instance
protected function escape_values( $values ) {
$escaped_values = [];
foreach ( $values as $value ) {
if ( is_null( $value ) ) {
Expand Down Expand Up @@ -278,9 +295,11 @@ protected function escape_string( $value ) {
* @return string
*/
protected function get_dump_comment( $comment ) {
$comment = str_replace( array( "\r\n", "\r" ), "\n", $comment );

return implode(
"\n",
array( '--', sprintf( '-- %s', $comment ), '--' )
array( '--', sprintf( '-- %s', str_replace( "\n", "\n-- ", $comment ) ), '--' )
);
}

Expand All @@ -292,18 +311,27 @@ protected function get_dump_comment( $comment ) {
* @return bool
*/
protected function table_has_records( $table_name ) {
$pdo = $this->get_pdo();
$stmt = $pdo->prepare( 'SELECT COUNT(*) FROM ' . $table_name );
$stmt->execute();
$table_name = $this->quote_identifier( $table_name );
$stmt = $this->get_sqlite_pdo()->query( 'SELECT COUNT(*) FROM ' . $table_name );
return $stmt->fetchColumn() > 0;
}

/**
* Get the PDO instance.
* Quote a MySQL identifier.
*
* @param string $identifier Identifier to quote.
* @return string
*/
protected function quote_identifier( $identifier ) {
return '`' . str_replace( '`', '``', $identifier ) . '`';
}

/**
* Get the underlying SQLite PDO instance.
*
* @return PDO
*/
protected function get_pdo() {
protected function get_sqlite_pdo() {
if ( $this->driver instanceof WP_SQLite_Translator ) {
return $this->driver->get_pdo();
}
Expand Down
Loading
Loading