Refactor backup runner and improve sync behavior

.gitignore

@@ -98,7 +98,7 @@ ipython_config.py
 #   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
 #   This is especially recommended for binary packages to ensure reproducibility, and is more
 #   commonly ignored for libraries.
-uv.lock
+# uv.lock
 
 # poetry
 #   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
@@ -199,7 +199,7 @@ cython_debug/
 #   that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
 #   and can be added to the global gitignore or merged into this file. However, if you prefer,
 #   you could uncomment the following to ignore the entire vscode folder
-# .vscode/
+.vscode/
 
 # Ruff stuff:
 .ruff_cache/
@@ -218,4 +218,8 @@ __marimo__/
 # Test data
 config
 backups
-start.sh
+start.sh
+
+.DS_Store
+.codex
+CLAUDE.md

Dockerfile

@@ -2,24 +2,22 @@
 FROM python:3.13-slim
 
 # Set work directory
-WORKDIR /home/docker
+WORKDIR /home/docker/github-backup
 
 # Install system dependencies including git
 RUN apt-get update && \
     apt-get install -y git && \
     rm -rf /var/lib/apt/lists/*
 
 # Copy the necessary files
-COPY github-backup/ ./github-backup/
-COPY pyproject.toml .
-COPY config.json.example ./github-backup/
-COPY backup.sh .
+COPY github_backup/ ./github_backup/
+COPY pyproject.toml README.md backup.sh config.json.example ./
 
 # Install project dependencies
 RUN pip install --no-cache-dir -e .
 
 # Set permissions
-RUN chmod -R 777 /home/docker && \
+RUN chmod -R 775 /home/docker && \
     chown -R 99:100 /home/docker
 
 # Use the non-root user to run the container

README.md

@@ -1,32 +1,78 @@
-# GitHub backup script
+# GitHub Backup
 
-This container contains a script, `backup.py`, for backing up GitHub repositories.
+This project builds a Docker image and Python CLI for backing up every GitHub repository visible to a personal access token. Repositories are stored as bare Git repositories under `{directory}/{owner}/{repo}`.
 
-The script requires a GitHub token and a destination directory. It then uses the token to populate the destination directory with clones of all the repositories the token can access.
+The current implementation is aimed at unattended runs on Unraid, but the backup engine is now packaged as a normal Python module as well.
 
-It is possible to set it to run on a schedule, and repeated runs only update the already existing backups and add new repositories, if any.
+## What It Does
 
-## Installation
+- Lists repositories available to the configured GitHub token.
+- Optionally appends repositories from explicitly configured extra orgs.
+- Optionally filters repositories by owner.
+- Clones new repositories as bare repositories.
+- Fetches updates for existing repositories.
+- Runs on a schedule inside the container when started with `backup.sh`.
 
-Installation can be completed via CA on Unraid.
+## Performance And Robustness Changes
 
-## Configuring
+- Uses `per_page=100` when listing repositories from the GitHub API.
+- Syncs repositories concurrently with a bounded worker pool.
+- Avoids rewriting the persistent config file on container startup.
+- Stops embedding the token in Git remote URLs.
+- Removes the expensive recursive `chown` after every run.
+- Logs Git fetch ref changes when a repo actually changed.
+- Suppresses per-repo no-op logs and keeps output focused on changed repos, failures, and the final summary.
 
-### Create a token
+## Configuration
 
-For authorization you need to create a new personal GitHub token. You can do this from [this](https://github.com/settings/tokens) page.
+`config.json` supports:
 
-![Step 1](https://raw.githubusercontent.com/lnxd/docker-github-backup/master/images/new-token-1.png)
+```json
+{
+  "token": "github_token_here",
+  "directory": "/home/docker/backups",
+  "concurrency": 4,
+  "schedule_seconds": 86400,
+  "failure_delay_seconds": 300,
+  "extra_orgs": ["optional-extra-org"],
+  "owners": ["optional-owner-filter"]
+}
+```
 
-When you click the **Generate new token** button you enter the token creation screen. Here you should give the token a descriptive name and choose its *scopes*, which basically determine what the token is allowed to do.
+Environment variables override config values:
 
-![Step 2](https://raw.githubusercontent.com/lnxd/docker-github-backup/master/images/new-token-2.png)
+- `TOKEN` or `GITHUB_TOKEN`
+- `BACKUP_DIRECTORY`
+- `CONCURRENCY`
+- `SCHEDULE`
+- `FAILURE_DELAY_SECONDS`
+- `EXTRA_ORGS`
+- `OWNERS`
 
-To backup public and private repositories you need to select only the **repo** scope. If you have no need for private repositories just choose the **public_repo** scope.
+`extra_orgs` adds repositories from named organizations on top of the normal `/user/repos` sync. This is useful when your token can access an org, but you want that org included explicitly without changing the default behavior for your own repos.
 
-![Step 3](https://raw.githubusercontent.com/lnxd/docker-github-backup/master/images/new-token-3.png)
+## Docker
 
-After clicking the **Generate token** button you're presented with the generated token. Remember to store it now, as GitHub won't show it to you anymore!
+Build the image:
 
-## Final notes
-If you notice any bugs, feel free to open an Issue or a pull request. For support with using this on Unraid, you can reach me best via the [support thread](https://forums.unraid.net/topic/104589-support-lnxd-phoenixminer-amd/) on the Unraid Community Forums.
+```bash
+docker build -t github-backup:latest .
+```
+
+Run it:
+
+```bash
+docker run --rm \
+  --name github-backup \
+  -e TOKEN=ghp_your_token_here \
+  -e SCHEDULE=86400 \
+  -v "$(pwd)/config:/home/docker/github-backup/config" \
+  -v "$(pwd)/backups:/home/docker/backups" \
+  github-backup:latest
+```
+
+## Local Usage
+
+```bash
+python -m github_backup ./config.json
+```

backup.sh

@@ -1,28 +1,21 @@
 #!/bin/sh
 
+set -eu
+
 echo "Project: github-backup"
 echo "Author:  Frazzer951"
 echo "Base:    Python 3.13-slim"
 echo "Target:  Unraid"
 echo ""
 
-# If config doesn't exist yet, create it
-if [ ! -f /home/docker/github-backup/config/config.json ]; then
-    cp /home/docker/github-backup/config.json.example /home/docker/github-backup/config/config.json
-fi
-
-# Update config.json
-cp /home/docker/github-backup/config/config.json /home/docker/github-backup/config.json
+APP_ROOT=/home/docker/github-backup
+CONFIG_DIR=${CONFIG_DIR:-$APP_ROOT/config}
+CONFIG_PATH=${CONFIG_PATH:-$CONFIG_DIR/config.json}
 
-# Update token in config.json match $TOKEN environment variable
-sed -i '/token/c\    \"token\" : \"'${TOKEN}'\",' /home/docker/github-backup/config.json
+mkdir -p "$CONFIG_DIR"
 
-# Return config.json to persistant volume
-cp /home/docker/github-backup/config.json /home/docker/github-backup/config/config.json
+if [ ! -f "$CONFIG_PATH" ]; then
+    cp "$APP_ROOT/config.json.example" "$CONFIG_PATH"
+fi
 
-# Start backup
-while true; do
-    python3 ./github-backup/github-backup.py /home/docker/github-backup/config/config.json
-    chown -R 99:100 /home/docker/backups
-    sleep $SCHEDULE
-done
+exec python3 -m github_backup "$CONFIG_PATH" --loop

config.json.example

@@ -1,4 +1,9 @@
 {
     "token": "0123456789abcdef0123456789abcdef",
-    "directory": "/home/docker/backups"
+    "directory": "/home/docker/backups",
+    "concurrency": 4,
+    "schedule_seconds": 86400,
+    "failure_delay_seconds": 300,
+    "extra_orgs": [],
+    "owners": []
 }

github_backup/__init__.py

@@ -0,0 +1,8 @@
+"""GitHub backup package."""
+
+from importlib.metadata import PackageNotFoundError, version
+
+try:
+    __version__ = version("github_backup")
+except PackageNotFoundError:
+    __version__ = "0.0.0"

github_backup/__main__.py

@@ -0,0 +1,4 @@
+from github_backup.cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())