Skip to content

Keep pipecd server data even when developer shut down pipecd or kind on local computer#6192

Merged
Warashi merged 5 commits into
pipe-cd:masterfrom
linhdangduy:kind-mount-volume-to-keep-storage-data-on-local
Sep 14, 2025
Merged

Keep pipecd server data even when developer shut down pipecd or kind on local computer#6192
Warashi merged 5 commits into
pipe-cd:masterfrom
linhdangduy:kind-mount-volume-to-keep-storage-data-on-local

Conversation

@linhdangduy
Copy link
Copy Markdown
Contributor

@linhdangduy linhdangduy commented Aug 29, 2025

What this PR does:

  • Create docker volume pipecd-data.
  • Mount above volume to local kind cluster, and use it for mysql, minio, cache deployment.

Why we need it:

On local, every time make stop/pipecd is executed, all the local data that the developer has "worked hard" to create on the local pipecd server also got lost, then she have to create them all over again on next pipecd run.
The root cause is: the mysql, minio, cache storage deployments got restart, and their data also lost.

This PR is for mounting the local volume to storage data in kind cluster, so the data will be kept even when the mysql, minio, cache storage deployments are shut down. When the developer executes make stop/pipecd, or make kind-down or even restarts her computer, she would still have all data available on the next run.

Which issue(s) this PR fixes:

Solve Item 3 in #6152

Does this PR introduce a user-facing change?:

  • How are users affected by this change: No
  • Is this breaking change: No
  • How to migrate (if breaking change):

Test

To test this PR's change, you can do the following step:

  1. turn off current kind cluster if there is: make kind-down
  2. make kind-up -> make run/pipecd -> port forward ... then access to pipecd web
  3. Create piped, or new api key
  4. make stop/pipecd -> make kind-down: Turn off pipecd/kind
  5. Run kind, pipecd again like step (2), then go to piped or api key manage page. We can see that the created ones on step (3) still be available.

To confirm the pipecd-data volume content:

 docker run -v $(docker volume inspect pipecd-data --format '{{ .Mountpoint }}'):/mount alpine ls /mount

# minio
# mysql
# redis

@linhdangduy linhdangduy changed the title Keep local storage data by mounting to local volumes Keep local storage data even when developer shut down pipecd or kind on local computer Aug 29, 2025
@linhdangduy linhdangduy changed the title Keep local storage data even when developer shut down pipecd or kind on local computer Keep pipecd server data even when developer shut down pipecd or kind on local computer Aug 29, 2025
@linhdangduy linhdangduy force-pushed the kind-mount-volume-to-keep-storage-data-on-local branch from b4c69ae to 2a916c8 Compare September 3, 2025 13:04
@linhdangduy linhdangduy marked this pull request as ready for review September 3, 2025 13:05
@linhdangduy linhdangduy requested a review from a team as a code owner September 3, 2025 13:05
Copy link
Copy Markdown
Member

@Okabe-Junya Okabe-Junya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice improvement!
left just a minor comment, PTAL :)

Comment thread hack/create-kind-cluster.sh Outdated
Signed-off-by: Dan <dangduylinh.bk@gmail.com>
Signed-off-by: Dan <dangduylinh.bk@gmail.com>
Signed-off-by: Dan <dangduylinh.bk@gmail.com>
Signed-off-by: Dan <dangduylinh.bk@gmail.com>
@linhdangduy linhdangduy force-pushed the kind-mount-volume-to-keep-storage-data-on-local branch from 0336766 to 940b309 Compare September 12, 2025 09:34
Okabe-Junya
Okabe-Junya previously approved these changes Sep 12, 2025
Copy link
Copy Markdown
Member

@Okabe-Junya Okabe-Junya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM!

…local

Signed-off-by: Dan <dangduylinh.bk@gmail.com>
@linhdangduy
Copy link
Copy Markdown
Contributor Author

@Okabe-Junya Thank you. I have just resolve a conflict with master branch. Please check it again 🙇🏼

Copy link
Copy Markdown
Member

@khanhtc1202 khanhtc1202 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement, thank you 🙌

Copy link
Copy Markdown
Member

@Warashi Warashi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏻

@Warashi Warashi merged commit 2cd6855 into pipe-cd:master Sep 14, 2025
42 checks passed
@linhdangduy linhdangduy deleted the kind-mount-volume-to-keep-storage-data-on-local branch September 15, 2025 01:54
Comment on lines +52 to +56
nodes:
- role: control-plane
extraMounts:
- hostPath: ${VOLUME_MOUNT_POINT}
containerPath: /tmp/pipecd-data
Copy link
Copy Markdown
Member

@khanhtc1202 khanhtc1202 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized this only auto creates volumes for Kind, and I'm using Docker4Mac k8s cluster on my local, so I have to do it myself 😂 cc @linhdangduy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants