This repo is used to illustrate a FluxCD-managed, GitOps-based Kubernetes cluster (k3s) on Oracle Cloud free tier. Currently it's just one master node but we use Tailscale to address the nodes so expanding the cluster across regions should be straightforward with the TailScale Kubernetes operator (see TailScale Github Action). It attempts modern cloud-native practices including declarative infrastructure management, automated deployments, and continuous reconciliation.
This repository implements security best practices to protect against GitHub PAT attacks, supply chain vulnerabilities, and cross-cloud lateral movement as documented in Wiz's security research.
📋 Security Policy - Security controls, threat model, and best practices 📊 Security Assessment - Detailed security audit and findings
- ✅ No PAT usage - Uses scoped
GITHUB_TOKENinstead of Personal Access Tokens - ✅ Pinned GitHub Actions - All actions pinned to commit SHAs to prevent supply chain attacks
- ✅ Automated secret scanning - Gitleaks integration for detecting leaked credentials
- ✅ Minimal permissions - All workflows follow principle of least privilege
- ✅ Dependency updates - Dependabot configured for automated security updates
- Add UI gimlet/capacitor-ui
- Implement podinfo microservice application
- Security Hardening: GitHub Actions security, secret scanning, supply chain protection
- Add Traefik ingress
- Add requesite configuration for Tailscale
- Multi-Environment Support: Add staging and production overlays
- Secrets Management: Implement SOPS or External Secrets Operator
- Monitoring Stack: Deploy Prometheus, Grafana, and AlertManager
- Image Update Automation: Automated container image updates
- Advanced Security: Network policies, Pod Security Standards, admission controllers
- Policy as Code: OPA Gatekeeper or Kyverno for compliance
- Disaster Recovery: Backup and restore procedures
- Advanced Observability: Distributed tracing and log aggregation
- Progressive Delivery:
clusters/devpoints to the oracle cluster- apps base configuration in
apps/base - apps override configuration in
apps/dev
This repository implements a GitOps workflow where:
- Git is the single source of truth for infrastructure and application definitions
- Flux CD automatically synchronizes cluster state with repository state
- Kustomize handles configuration management across environments
- OCI repositories provide artifact-based deployments for applications
┌──────────────────┐ ┌──────────────────┐ ┌─────────────────────┐
│ Git Repository │──── │ Flux CD │────│ Oracle K3s │
│ (This Repo) │ │ Controllers │ │ Cluster │
└──────────────────┘ └──────────────────┘ └─────────────────────┘
│ │ │
│ │ │
┌────▼────┐ ┌────▼────┐ ┌────▼────┐
│ Apps │ │ Source │ │ Running │
│ Config │ │ Control │ │ Workload│
└─────────┘ └─────────┘ └─────────┘
Diagram
Note
Generated diagram using KubeDiagram and Github Action
- Oracle Cloud Account with free tier K3s cluster provisioned
- kubectl configured to access your cluster
- Flux CLI installed (installation guide)
- Git access to this repository
-
Fork repo
-
Bootstrap Flux on your cluster:
export GITHUB_USER=<github username> # preferably a token scoped to your repo export GITHUB_TOKEN=<github token> # Install Flux components flux install # Create the GitRepository source flux bootstrap github \ --owner <github username> --repository <destination github repo> --private-key-file=/path/to/your/ssh/key \ --personal
-
Verify deployment:
# Check Flux components flux get sources git flux get kustomizations # Monitor reconciliation flux logs --follow
flux-infra/
├── .github/
│ └── workflows/ # CI/CD automation
│ ├── claude-code-review.yml # AI-powered code review
│ ├── e2e.yaml # End-to-end testing workflow
│ ├── kustomize-verify.yml # Validates Kustomize builds
│ └── tailscale.yml # Tailscale integration workflow
├── .vscode/
│ └── settings.json # VSCode project settings
├── apps/
│ ├── base/ # Base application configurations
│ │ ├── capacitor-ui/ # Capacitor UI application
│ │ │ ├── flux-kustomization.yaml # Flux deployment config
│ │ │ ├── kustomization.yaml # Kustomize base config
│ │ │ └── oci-repository.yaml # OCI source definition
│ │ └── podinfo/ # Podinfo microservice
│ │ ├── helm-release.yaml # Helm release definition
│ │ ├── helm-repository.yaml # Helm repository source
│ │ └── kustomization.yaml # Kustomize base config
│ └── dev/ # Development environment overlays
│ ├── capacitor-ui/
│ │ └── kustomization.yaml # Dev overlay for Capacitor UI
│ ├── podinfo/
│ │ ├── helm-release-values.yaml # Dev-specific Helm values
│ │ └── kustomization.yaml # Dev overlay for Podinfo
│ └── kustomization.yaml # Root dev apps config
├── clusters/
│ └── dev/ # Development cluster configuration
│ ├── apps/ # Application deployment configs
│ │ ├── flux-kustomization-apps.yaml # Apps Flux kustomization
│ │ └── kustomization.yaml # Apps orchestration
│ ├── infra/ # Infrastructure components
│ │ ├── configmap-env.yaml # Environment configuration
│ │ ├── flux-kustomization.yaml # Infrastructure Flux config
│ │ ├── git-repository.yaml # Git source definition
│ │ ├── gotk-components.yaml # Flux core components
│ │ └── kustomization.yaml # Infrastructure orchestration
│ └── kustomization.yaml # Root cluster config
├── scripts/
│ └── validate.sh # Validation script
├── .gitignore # Git ignore patterns
├── flux-init.sh # Flux initialization script
├── README.md # This file
clusters/dev/kustomization.yaml
├── infra/ # Infrastructure first
│ ├── Flux controllers
│ └── Core services
└── apps/ # Applications second
└── capacitor-ui/ # UI application
└── → apps/base/capacitor-ui/
- Flux CD Controllers: Source, Kustomize, and Image controllers
- Git Synchronization: Automatic sync with this repository every 10 minutes
- Kustomize Integration: Configuration management and environment overlays
- Capacitor UI: Kubernetes dashboard and management interface
- Source: OCI registry (
ghcr.io/gimlet-io/capacitor-manifests) - Deployment:
flux-systemnamespace - Updates: Semantic versioning (
>=0.1.0)
- Source: OCI registry (
-
Create a feature branch:
git checkout -b feature/add-monitoring
-
Make your changes to the appropriate directory:
- Infrastructure changes:
clusters/dev/infra/ - Application changes:
apps/base/orapps/dev/
- Infrastructure changes:
-
Test locally (requires kubectl and kustomize):
# Validate Kustomize builds kubectl kustomize ./clusters/dev # Check for any issues kubectl kustomize ./clusters/dev --enable-helm
-
Commit and push:
git add . git commit -m "feat: add monitoring stack" git push origin feature/add-monitoring
-
Create Pull Request - CI will automatically validate your changes
Automated Checks:
- Kustomize Validation: Ensures all Kustomize configurations build successfully
- Kubernetes Schema Validation: Validates resources against K8s API schemas
- Security Scanning: Checks for security best practices (planned)
Deployment:
- Automatic: Flux monitors the
mainbranch and deploys changes within 10 minutes - Manual: Force sync with
flux reconcile kustomization flux-system
# Overall system health
flux get all
# Specific component status
flux get sources git
flux get kustomizations
# View recent events
flux events
# Follow logs
flux logs --follow# Check Kustomization status
kubectl describe kustomization capacitor -n flux-system
# View application pods
kubectl get pods -n flux-system
# Check resource events
kubectl get events -n flux-system --sort-by='.lastTimestamp'Issue: Kustomization stuck in "Progressing" state
# Solution: Check for dependency issues
kubectl describe kustomization <name> -n flux-systemIssue: OCI repository authentication failures
# Solution: Verify OCI repository accessibility
flux get sources ociIssue: Git repository sync failures
# Solution: Check SSH key configuration
kubectl get secret flux-system -n flux-system -o yaml- Declarative Configuration: Everything defined as code
- Version Controlled: All changes tracked in Git
- Automated Synchronization: Flux handles deployment reconciliation
- Rollback Capability: Git history enables easy rollbacks
- Namespace Isolation: Logical separation of components
- Resource Quotas: Prevent resource exhaustion (planned)
- Network Policies: Micro-segmentation for security (planned)
- Health Checks: Ensure application availability
- Free Tier Limits: Optimized for 2 OCPU, 12GB RAM constraints
- ARM Architecture: Compatible with ARM64 workloads
- Regional Deployment: Single availability domain deployment
- Cost Optimization: Resource requests tuned for free tier
Security is a top priority for this project. We've implemented comprehensive security controls to protect against:
- GitHub PAT attacks and credential theft
- Supply chain attacks targeting GitHub Actions
- Secret exposure in code repositories
- Cross-cloud lateral movement from GitHub to cloud infrastructure
For detailed information, see:
- SECURITY.md - Security policy, controls, and vulnerability reporting
- SECURITY_ASSESSMENT.md - Complete security audit report
DO NOT open public issues for security vulnerabilities. Instead, email repository maintainers with details.
- Fork this repository
- Set up your own Oracle Cloud K3s cluster for testing
- Bootstrap Flux pointing to your fork
- Make changes and test them
- Submit a Pull Request with clear description
Before submitting a PR, ensure:
- ✅ No secrets or credentials committed
- ✅ GitHub Actions pinned to commit SHAs
- ✅ Minimal workflow permissions requested
- ✅ Security implications documented
- ✅ Pre-commit validation passes
- Kustomize: Use proper base/overlay patterns
- Naming: Follow Kubernetes naming conventions
- Documentation: Update README for significant changes
- Testing: Ensure all Kustomize builds succeed
- All changes require Pull Request review
- CI validation must pass
- Test in development environment first
- Consider impact on running workloads
This project uses validation scripts adapted from FluxCD's flux2-kustomize-helm-example.
Run validation before committing changes:
./scripts/validate.sh
## 📄 License
This project is provided as-is for demonstration and learning purposes.
---
**Questions or Issues?**
- Check the [troubleshooting section](#-monitoring-and-troubleshooting)
- Review [Flux CD documentation](https://fluxcd.io/flux/)
- Open an issue in this repository