-
Notifications
You must be signed in to change notification settings - Fork 28
Add ecosystem component: pg_profile & pg_repack #271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
cb3865c
add: pg_profile
hs-liuxh c2d7b06
add: pg_repack
hs-liuxh 0037e4f
Fix: missing compatible extensions in the sidebar
hs-liuxh b8f8e75
Fix some typos in pg_profile & pg_repack
hs-liuxh 5e29ae8
Sync IvorySQL commit "Replace the pg_dialect control parameter with a…
hs-liuxh 3b5d292
Merge branch 'IvorySQL:master' into master
hs-liuxh d5d1261
Trigger github action.
hs-liuxh 8c5638d
Merge branch 'master' into master
hs-liuxh 7b34805
Update pg_profile.adoc
hs-liuxh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
105 changes: 105 additions & 0 deletions
105
CN/modules/ROOT/pages/master/ecosystem_components/pg_profile.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
|
|
||
| :sectnums: | ||
| :sectnumlevels: 5 | ||
|
|
||
| = pg_profile | ||
|
|
||
| == 概述 | ||
| pg_profile 是一个用于 PostgreSQL 数据库性能分析的扩展工具,主要用于统计目标数据库中的资源密集型活动,帮助用户深入了解数据库运行状态、找出性能瓶颈并进行优化。它的报告本质上是“两个采样点的差量分析”,得出采样间隔内的增量负载,采样点序号从 1 开始计数。 | ||
|
|
||
| pg_profile 硬依赖 dblink 与 plpgsql(其自身完全由 SQL 和 PL/pgSQL 编写,无需任何外部库或软件);此外,建议安装 pg_stat_statements 扩展,以便在报告中获取 SQL 语句级别的统计信息。pg_profile 的版本与 PostgreSQL 的版本强相关,当前最高支持到 PG 18,具体的版本支持情况请查看 pg_profile 官方 github 仓库的 README. | ||
|
|
||
| IvorySQL 的 PG 模式和 Oracle 兼容模式都已经适配 pg_profile。 | ||
|
|
||
| 项目地址:<https://github.com/zubkov-andrei/pg_profile> | ||
|
|
||
| 开源协议:PostgreSQL License | ||
|
|
||
| == 安装启用 | ||
|
|
||
| === 源码编译 | ||
| 执行源码编译和安装: | ||
| [literal] | ||
| ---- | ||
| # 构建并安装 pg_profile 及其依赖扩展 | ||
| make -C contrib/dblink install | ||
| make -C contrib/pg_stat_statements install | ||
| make -C contrib/pg_profile install | ||
| ---- | ||
|
|
||
| 安装产物为 `pg_profile.control` 与 `pg_profile--4.11.sql`。 | ||
|
|
||
| === 修改配置 | ||
| pg_profile 依赖的 pg_stat_statements 必须通过 `shared_preload_libraries` 在服务启动时加载。 | ||
|
|
||
| 编辑 `ivorysql.conf` ,在 `shared_preload_libraries` 配置项末尾追加 `pg_stat_statements`: | ||
| [literal] | ||
| ---- | ||
| # ivorysql.conf | ||
| shared_preload_libraries = 'liboracle_parser, ivorysql_ora, pg_stat_statements' | ||
| ---- | ||
|
|
||
| === 重启服务 | ||
| [literal] | ||
| ---- | ||
| pg_ctl -D $PGDATA restart | ||
| ---- | ||
|
|
||
| === 安装扩展 | ||
| PG 模式与 Oracle 模式会话下命令相同: | ||
| [literal] | ||
| ---- | ||
| CREATE SCHEMA profile; | ||
| CREATE SCHEMA dblink; | ||
| CREATE SCHEMA statements; | ||
| CREATE EXTENSION dblink SCHEMA dblink; | ||
| CREATE EXTENSION pg_stat_statements SCHEMA statements; | ||
| CREATE EXTENSION pg_profile SCHEMA profile; | ||
|
|
||
| SELECT extname, extversion FROM pg_extension WHERE extname = 'pg_profile'; | ||
| extname | extversion | ||
| ------------+------------ | ||
| pg_profile | 4.11 | ||
| ---- | ||
|
|
||
| == 使用流程 | ||
|
|
||
| === 创建样本 | ||
| [literal] | ||
| ---- | ||
| -- 第一次采样 | ||
| SELECT * FROM profile.take_sample(); | ||
| server | result | elapsed | ||
| --------+--------+------------- | ||
| local | OK | 00:00:01.34 | ||
|
|
||
| -- ……业务负载运行一段时间后,再次采样 | ||
| SELECT * FROM profile.take_sample(); | ||
| ---- | ||
|
|
||
| 生产环境通常用定时任务周期采样(如 30 分钟一次),可配合 pg_cron 或外部 crontab: | ||
| [literal] | ||
| ---- | ||
| */30 * * * * psql -c 'SELECT profile.take_sample()' >/dev/null | ||
| ---- | ||
|
|
||
| === 查看样本 | ||
| [literal] | ||
| ---- | ||
| SELECT sample, sample_time, sizes_collected FROM profile.show_samples(); | ||
| sample | sample_time | sizes_collected | ||
| --------+------------------------+----------------- | ||
| 1 | 2026-06-12 09:00:00+00 | t | ||
| 2 | 2026-06-12 09:30:00+00 | t | ||
| ---- | ||
|
|
||
| === 生成报告 | ||
| [literal] | ||
| ---- | ||
| -- 生成样本 1 到样本 2 之间的负载报告(HTML 文本) | ||
| \o report_1_2.html | ||
| SELECT profile.get_report(1, 2); | ||
| \o | ||
| ---- | ||
|
|
||
| 以上函数在 Oracle 模式会话(Oracle 端口/1521)中调用方式与结果完全一致。 | ||
102 changes: 102 additions & 0 deletions
102
CN/modules/ROOT/pages/master/ecosystem_components/pg_repack.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
|
|
||
| :sectnums: | ||
| :sectnumlevels: 5 | ||
|
|
||
| = pg_repack | ||
|
|
||
| == 概述 | ||
| pg_repack 是 PostgreSQL 的一个扩展,用于在线重整表和索引、消除存储膨胀。它能达到与 `VACUUM FULL` 类似的效果,回收空间,但几乎不阻塞业务读写。 | ||
|
|
||
| IvorySQL 的 PG 模式和 Oracle 兼容模式都已经适配 pg_repack。 | ||
|
|
||
| 项目地址:<https://github.com/reorg/pg_repack> | ||
|
|
||
| 官方文档:<https://reorg.github.io/pg_repack/> | ||
|
|
||
| 开源协议:BSD-3-Clause License | ||
|
|
||
| == 原理 | ||
| VACUUM FULL 因没有"重写期间的并发变更捕获机制",重写整表时全程持有 ACCESS EXCLUSIVE 锁,期间表不可读写,锁时长随表增大。pg_repack 通过"影子表 + 触发器 + 文件节点交换"把强锁压缩到首尾两个瞬间,从而支持在线重整。 | ||
|
|
||
| === 变更捕获与数据拷贝 | ||
| * 先在原表上安装行级触发器,把此后发生的 INSERT/UPDATE/DELETE 记入日志表(repack.log_<oid>)——这一步需短暂的 ACCESS EXCLUSIVE 锁以原子地装上触发器,装好即释放; | ||
|
|
||
| * 创建与原表结构相同的影子表(repack.table_<oid>),在一致性 MVCC 快照下把原表数据批量拷入;此阶段只持 SHARE UPDATE EXCLUSIVE,原表照常读写; | ||
|
|
||
| * 在影子表上重建索引,并回放日志,使影子表追平原表最新状态。 | ||
|
|
||
| === 交换与清理 | ||
| * 取得短暂的 ACCESS EXCLUSIVE 锁,回放剩余日志后,在系统目录层面交换原表与影子表的物理文件节点(relfilenode),表的 OID 保持不变——交换是瞬时的; | ||
|
|
||
| * 删除触发器、日志表,并删除影子表(此时它已指向旧堆,删除即回收旧物理文件);最后对表执行 ANALYZE。 | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| 使用前提:目标表必须有主键或非空唯一键;操作过程中约需要两倍表大小的临时磁盘空间。 | ||
| ==== | ||
|
|
||
| == 安装启用 | ||
| 环境中已经安装了 IvorySQL5 及以上版本。pg_repack 由两部分组成:服务端扩展(`pg_repack.so` 与 SQL 脚本)和客户端命令行工具 `pg_repack`。 | ||
|
|
||
| 假设当前已安装 IvorySQL5,并且 pg_config 工具可用。 | ||
|
|
||
| === 源码安装 | ||
| 拉取 pg_repack 源码: | ||
| [literal] | ||
| ---- | ||
| git clone --branch ver_1.5.3 https://github.com/reorg/pg_repack.git | ||
| ---- | ||
|
|
||
| 执行编译和安装: | ||
| [literal] | ||
| ---- | ||
| cd pg_repack | ||
| make | ||
| make install | ||
| ---- | ||
|
|
||
| === 创建扩展 | ||
| [literal] | ||
| ---- | ||
| [ivorysql@localhost ivorysql]$ psql | ||
| psql (18.0) | ||
| Type "help" for help. | ||
|
|
||
| ivorysql=# CREATE EXTENSION pg_repack; | ||
| CREATE EXTENSION | ||
| ---- | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| `CREATE EXTENSION pg_repack` 需要超级用户执行;客户端工具 `pg_repack` 默认也要求以超级用户身份连接运行。 | ||
| ==== | ||
|
|
||
| == 使用流程 | ||
| pg_repack 通过命令行客户端驱动,连接参数与 psql 一致(`-h` 主机、`-p` 端口、`-U` 用户、`-d` 数据库,或使用 `PGHOST` / `PGPORT` / `PGUSER` 环境变量)。 | ||
|
|
||
| === 重整整个数据库中所有符合条件的表 | ||
| [literal] | ||
| ---- | ||
| pg_repack -d ivorysql | ||
| ---- | ||
|
|
||
| === 重整指定表 | ||
| 推荐使用 schema.table 形式: | ||
| [literal] | ||
| ---- | ||
| pg_repack -d ivorysql -t public.big_table | ||
| ---- | ||
|
|
||
| === 仅重建索引 | ||
| [literal] | ||
| ---- | ||
| pg_repack -d ivorysql -t big_table --only-indexes # 重建该表的全部索引 | ||
| pg_repack -d ivorysql -i public.big_table_idx # 重建单个索引 | ||
| ---- | ||
|
|
||
| === 预演模式 | ||
| 只显示将要执行的动作,不实际重整: | ||
| [literal] | ||
| ---- | ||
| pg_repack -d ivorysql -t big_table --dry-run | ||
| ---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
106 changes: 106 additions & 0 deletions
106
EN/modules/ROOT/pages/master/ecosystem_components/pg_profile.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
|
|
||
| :sectnums: | ||
| :sectnumlevels: 5 | ||
|
|
||
| = pg_profile | ||
|
|
||
| == Overview | ||
| pg_profile is an extension tool for PostgreSQL database performance analysis. It is mainly used to collect resource-intensive activities in a target database, helping users gain in-depth insight into database runtime status, identify performance bottlenecks, and perform optimization. Its report is essentially a "delta analysis between two sample points", deriving the incremental load within the sampling interval, with sample point sequence numbers starting from 1. | ||
|
|
||
| pg_profile hard-depends on two extensions: dblink and plpgsql (it is written entirely in SQL and PL/pgSQL and requires no external libraries or software). Additionally, it is recommended to install the pg_stat_statements extension to obtain SQL statement-level statistics in the reports. The version of pg_profile is strongly tied to the version of PostgreSQL. For the specific version support, please refer to the README in the official pg_profile github repository. | ||
|
|
||
| Both the PG mode and Oracle compatibility mode of IvorySQL have been adapted for pg_profile. | ||
|
|
||
| Project address: <https://github.com/zubkov-andrei/pg_profile> | ||
|
|
||
| License: PostgreSQL License | ||
|
|
||
| == Installation and Enabling | ||
|
|
||
| === Source Code Compilation | ||
|
|
||
| Perform source code compilation and installation: | ||
| [literal] | ||
| ---- | ||
| # Build and install pg_profile and its dependent extensions | ||
| make -C contrib/dblink install | ||
| make -C contrib/pg_stat_statements install | ||
| make -C contrib/pg_profile install | ||
| ---- | ||
|
|
||
| The installation artifacts are `pg_profile.control` and `pg_profile--4.11.sql`. | ||
|
|
||
| === Modify Configuration | ||
| The pg_stat_statements that pg_profile depends on must be loaded at server startup via `shared_preload_libraries`. | ||
|
|
||
| Edit `ivorysql.conf` and append `pg_stat_statements` to the end of the `shared_preload_libraries` configuration item: | ||
| [literal] | ||
| ---- | ||
| # ivorysql.conf | ||
| shared_preload_libraries = 'liboracle_parser, ivorysql_ora, pg_stat_statements' | ||
| ---- | ||
|
|
||
| === Restart the Service | ||
| [literal] | ||
| ---- | ||
| pg_ctl -D $PGDATA restart | ||
| ---- | ||
|
|
||
| === Install the Extension | ||
| The commands are identical in both PG mode and Oracle mode sessions: | ||
| [literal] | ||
| ---- | ||
| CREATE SCHEMA profile; | ||
| CREATE SCHEMA dblink; | ||
| CREATE SCHEMA statements; | ||
| CREATE EXTENSION dblink SCHEMA dblink; | ||
| CREATE EXTENSION pg_stat_statements SCHEMA statements; | ||
| CREATE EXTENSION pg_profile SCHEMA profile; | ||
|
|
||
| SELECT extname, extversion FROM pg_extension WHERE extname = 'pg_profile'; | ||
| extname | extversion | ||
| ------------+------------ | ||
| pg_profile | 4.11 | ||
| ---- | ||
|
|
||
| == Usage Workflow | ||
|
|
||
| === Create Samples | ||
| [literal] | ||
| ---- | ||
| -- First sampling | ||
| SELECT * FROM profile.take_sample(); | ||
| server | result | elapsed | ||
| --------+--------+------------- | ||
| local | OK | 00:00:01.34 | ||
|
|
||
| -- ...after the business workload runs for a while, sample again | ||
| SELECT * FROM profile.take_sample(); | ||
| ---- | ||
|
|
||
| In production environments, periodic sampling via scheduled tasks is typically used (e.g., once every 30 minutes), which can be combined with pg_cron or an external crontab: | ||
| [literal] | ||
| ---- | ||
| */30 * * * * psql -c 'SELECT profile.take_sample()' >/dev/null | ||
| ---- | ||
|
|
||
| === View Samples | ||
| [literal] | ||
| ---- | ||
| SELECT sample, sample_time, sizes_collected FROM profile.show_samples(); | ||
| sample | sample_time | sizes_collected | ||
| --------+------------------------+----------------- | ||
| 1 | 2026-06-12 09:00:00+00 | t | ||
| 2 | 2026-06-12 09:30:00+00 | t | ||
| ---- | ||
|
|
||
| === Generate Reports | ||
| [literal] | ||
| ---- | ||
| -- Generate the workload report between sample 1 and sample 2 (HTML text) | ||
| \o report_1_2.html | ||
| SELECT profile.get_report(1, 2); | ||
| \o | ||
| ---- | ||
|
|
||
| The above functions are invoked and produce identical results in an Oracle mode session (Oracle port/1521). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.