CICD#10
Conversation
…yment, Stripe worker deployment, website deployment, and weekly blog post generation
…ject acknowledgements
|
@William-W-Chen is attempting to deploy a commit to the bytebase Team on Vercel. A member of the Team first needs to authorize it. |
Greptile SummaryThis PR adds container and Helm deployment support plus a new schema migration workflow. The main changes are:
Confidence Score: 2/5This should not be merged until the build, container, and migration configuration paths are fixed.
Focus on
|
| Filename | Overview |
|---|---|
| server/services/migration-service.ts | Adds migration planning and apply flow using metadata, git checkout, temp plan files, and pgschema. |
| server/services/metadata-service.ts | Adds _pgconsole metadata CRUD for schema source configuration. |
| server/lib/git.ts | Adds cached git checkout sync for schema repositories. |
| Dockerfile | Bundles pgschema, installs git, and switches the runtime image to a non-root user. |
| helm-chart/templates/configmap.yaml | Mounts the pgconsole TOML configuration from chart values. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
UI[MigrationPanel] --> RPC[MigrationService]
RPC --> DB[_pgconsole metadata]
RPC --> Git[syncRepo checkout]
RPC --> PgSchema[pgschema plan/apply]
PgSchema --> PlanStore[plan-store temp JSON]
PgSchema --> Postgres[(PostgreSQL)]
Comments Outside Diff (1)
-
server/lib/config.ts, line 20-36 (link)Declare schema source
loadConfigFromStringnow assignsschema_sourceonto each connection, and the new tests readconns[0].schema_source, butConnectionConfigdoes not declare that property. A normal TypeScript build can fail with property/excess-field errors before the server bundle is produced.
Reviews (1): Last reviewed commit: "docs: update README.md for improved inst..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const details = getConnectionDetails(conn) | ||
| const schemaSource = await getSchemaSource(details, user?.email) | ||
| if (!schemaSource) { | ||
| throw new ConnectError( | ||
| 'No schema_source configured. Use SetMetadata to store a schema_source entry in the _pgconsole table.', | ||
| Code.FailedPrecondition, | ||
| ) |
There was a problem hiding this comment.
The config parser and example TOML now support connections.schema_source, but this path only calls getSchemaSource, which reads the _pgconsole metadata table. A deployment that follows the documented TOML configuration will still receive No schema_source configured and cannot plan migrations until the same data is duplicated through metadata.
| ARG TARGETARCH | ||
| RUN unzip bin/pgschema-linux-${TARGETARCH}.zip -d /usr/local/bin \ | ||
| && mv /usr/local/bin/pgschema-*-linux-${TARGETARCH} /usr/local/bin/pgschema \ | ||
| && chmod +x /usr/local/bin/pgschema |
There was a problem hiding this comment.
The builder stage is based on Alpine and does not install unzip before using it to extract the bundled pgschema archive. Docker builds can fail at this step with unzip: not found, so the GHCR image cannot be produced.
| ARG TARGETARCH | |
| RUN unzip bin/pgschema-linux-${TARGETARCH}.zip -d /usr/local/bin \ | |
| && mv /usr/local/bin/pgschema-*-linux-${TARGETARCH} /usr/local/bin/pgschema \ | |
| && chmod +x /usr/local/bin/pgschema | |
| ARG TARGETARCH | |
| RUN apk add --no-cache unzip \ | |
| && unzip bin/pgschema-linux-${TARGETARCH}.zip -d /usr/local/bin \ | |
| && mv /usr/local/bin/pgschema-*-linux-${TARGETARCH} /usr/local/bin/pgschema \ | |
| && chmod +x /usr/local/bin/pgschema |
| ENV PORT=9876 | ||
| EXPOSE 9876 | ||
|
|
||
| USER pgconsole |
There was a problem hiding this comment.
The image now runs as the pgconsole user, but the entrypoint writes PGCONSOLE_CONFIG to /etc/pgconsole.toml. That path is root-owned in this image, so containers started with env-based config fail before Node starts with a permission error.
| ) | ||
| } | ||
| } | ||
|
|
||
| export const metadataServiceHandlers: ServiceImpl<typeof MetadataService> = { | ||
| async initMetadataTable(req, context) { | ||
| if (!req.connectionId) { |
There was a problem hiding this comment.
All metadata operations use the unqualified _pgconsole table and the existence check only matches relname. In databases with a custom search_path, pgconsole can create the table in one schema and later read another schema's _pgconsole; a writable schema earlier in the search path could also shadow the metadata table and change the schema_source used for migrations.
| export async function syncRepo(connectionId: string, repo: string, branch?: string): Promise<{ commitHash: string }> { | ||
| // Serialize concurrent sync requests for the same connection | ||
| const existing = syncLocks.get(connectionId) | ||
| if (existing) { | ||
| return existing | ||
| } |
There was a problem hiding this comment.
The in-flight sync lock is keyed only by connectionId. If one request starts syncing main and the schema source is changed to another branch or repo before a second request arrives, the second request returns the first promise and plans against the old source instead of the newly requested one.
| if (exists) { | ||
| const cachedUrl = repoDirUrls.get(repoDir) | ||
| if (cachedUrl && cachedUrl !== repo) { | ||
| await rm(repoDir, { recursive: true, force: true }) | ||
| } | ||
| } | ||
|
|
||
| const stillExists = await access(join(repoDir, '.git')).then(() => true).catch(() => false) | ||
|
|
||
| if (stillExists) { | ||
| await exec('git', ['fetch', 'origin', ...(branch ? [branch] : [])], repoDir) | ||
| await exec('git', ['reset', '--hard', branch ? `origin/${branch}` : 'FETCH_HEAD'], repoDir) |
There was a problem hiding this comment.
Repo URL changes are detected only through the in-memory repoDirUrls map. After a process restart with an existing checkout on disk, the map is empty, so changing schema_source.repo for the same connection reuses the old checkout and fetches the old origin instead of replacing it with the new repository.
| const repoDir = getRepoDir(req.connectionId) | ||
| const schemaFilePath = validateSchemaPath(repoDir, schemaPath) | ||
| const outputJsonPath = join(tmpdir(), `pgconsole-plan-${randomUUID()}.json`) |
There was a problem hiding this comment.
Each plan writes a JSON file under /tmp, but plan removal and TTL eviction only delete the in-memory map entry. Repeated planning leaves DDL plan files behind for the lifetime of the container or host cleanup, which can consume disk and retain schema details longer than the 30-minute plan lifetime.
| data: | ||
| pgconsole.toml: | | ||
| {{- .Values.config | nindent 4 }} |
There was a problem hiding this comment.
The chart renders the entire pgconsole.toml into a ConfigMap, while the provided values and README examples include database passwords in that config. Anyone with ConfigMap read access in the namespace can read those credentials; this should be modeled as a Secret or split so sensitive values are mounted from Secrets.
2 participants
No description provided.