Skip to content
This repository was archived by the owner on Jun 9, 2025. It is now read-only.

codeinsights-db: correct data volume path#1936

Merged
emidoots merged 3 commits into
masterfrom
sg/fix-insights-data
Feb 26, 2021
Merged

codeinsights-db: correct data volume path#1936
emidoots merged 3 commits into
masterfrom
sg/fix-insights-data

Conversation

@emidoots
Copy link
Copy Markdown
Member

@emidoots emidoots commented Feb 26, 2021

I failed to verify /data was the correct path (it is for our other Postgres images, but not for this TimescaleDB image), which means codeinsights-db data is not being retained currently. I noticed this because the DB was lost on sourcegraph.com (chart looks empty.)

The good news is that I am actively working on building historical data for insights, so this data loss doesn't really matter - it'll all be rebuilt later.

Fixes https://github.com/sourcegraph/sourcegraph/issues/18704

Signed-off-by: Stephen Gutekanst stephen@sourcegraph.com

Checklist

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
Copy link
Copy Markdown
Contributor

@ggilmore ggilmore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slimsag A thought - why was the deployment still healthy if the path was incorrect?

Also, does this mean that existing users will start running into ephemeral storage issues soon?

@emidoots
Copy link
Copy Markdown
Member Author

A thought - why was the deployment still healthy if the path was incorrect?

It was able to write into /var/lib/postgresql/data with no problem (ephemeral storage) - I can't see any easy way to pick this up in a health check, but maybe you have an idea?

When the container restarted, it's as if the DB was entirely empty so "there is no issue" either.

Also, does this mean that existing users will start running into ephemeral storage issues soon?

Luckily this is unlikely, codeinsights is behind a feature flag and of-by-default. Plus, even when enabled the database is rather small unless you add a lot of insights and it records data for a really long time (like a year) - people will upgrade before they encounter any ephemeral storage issues I expect.

Stephen Gutekanst and others added 2 commits February 26, 2021 15:41
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

insights: data on sourcegraph.com lost due to container restart / bad volume mapping

2 participants