🚀 Kubernetes Monitoring Solution

A complete Kubernetes monitoring solution deployed on AWS using Terraform. This project provides comprehensive observability with Prometheus, Grafana, and an EFK (Elasticsearch, Fluentd, Kibana) stack - all deployed using Terraform.

✨ Features

• 🔄 Fully automated infrastructure provisioning with Terraform
• 🛡️ Secure RKE2 Kubernetes cluster on AWS EC2
• 📊 Prometheus & Grafana monitoring stack (via Terraform Helm provider)
• 📝 EFK logging stack (via Terraform Helm provider)
• 🔐 TLS encryption with Let's Encrypt certificates
• 🌐 Ingress Controller for external service access
• 🧩 Modular design with reusable Terraform modules

🏗️ Architecture

This solution implements a comprehensive Kubernetes monitoring and logging architecture with the following key components:

┌───────────────────────────────────────────────────────────────────────────────┐
│                                AWS Cloud                                       │
│                                                                               │
│  ┌────────────────────────────────────────────────────────────────────────┐   │
│  │                          VPC / Networking                              │   │
│  │                                                                        │   │
│  │  ┌───────────────────┐         ┌───────────────────┐                   │   │
│  │  │   Master Node     │         │   Worker Node(s)  │                   │   │
│  │  │                   │         │                   │                   │   │
│  │  │  ┌─────────────┐  │         │ ┌─────────────┐   │                   │   │
│  │  │  │ Control     │  │         │ │ Application │   │                   │   │
│  │  │  │ Plane       │  │         │ │ Workloads   │   │                   │   │
│  │  │  └─────────────┘  │         │ └─────────────┘   │                   │   │
│  │  └───────────────────┘         └───────────────────┘                   │   │
│  │                                                                        │   │
│  │                        RKE2 Kubernetes Cluster                         │   │
│  │                                                                        │   │
│  │  ┌─────────────────────────────┐    ┌─────────────────────────────┐   │   │
│  │  │      Monitoring Stack       │    │       Logging Stack         │   │   │
│  │  │                             │    │                             │   │   │
│  │  │  ┌─────────┐  ┌─────────┐   │    │  ┌─────────┐  ┌─────────┐   │   │   │
│  │  │  │Prometheus│  │ Grafana │   │    │  │Elastic- │  │ Kibana  │   │   │   │
│  │  │  │Operator  │  │ + Dash  │   │    │  │search   │  │ + Dash  │   │   │   │
│  │  │  └─────────┘  └─────────┘   │    │  └─────────┘  └─────────┘   │   │   │
│  │  │                             │    │        │                    │   │   │
│  │  │  ┌─────────┐  ┌─────────┐   │    │        ▼                    │   │   │
│  │  │  │AlertMgr │  │ Pushgtw │   │    │  ┌─────────┐                │   │   │
│  │  │  │         │  │         │   │    │  │Fluentd  │                │   │   │
│  │  │  └─────────┘  └─────────┘   │    │  │DaemonSet│                │   │   │
│  │  └─────────────────────────────┘    └─────────────────────────────┘   │   │
│  │                                                                        │   │
│  │  ┌─────────────────────────────┐                                      │   │
│  │  │    Ingress & Security       │     ┌─────────────────────────────┐  │   │
│  │  │                             │     │                             │  │   │
│  │  │  ┌─────────┐  ┌─────────┐   │     │   Terraform Infrastructure   │  │   │
│  │  │  │ Nginx   │  │ Cert-   │   │     │         Management          │  │   │
│  │  │  │ Ingress │  │ Manager │   │     │                             │  │   │
│  │  │  └─────────┘  └─────────┘   │     │  • Infrastructure Modules   │  │   │
│  │  │                             │     │  • Helm Releases            │  │   │
│  │  └─────────────────────────────┘     │  • Configuration Management │  │   │
│  │                                      └─────────────────────────────┘  │   │
│  └────────────────────────────────────────────────────────────────────────┘   │
│                                                                               │
└───────────────────────────────────────────────────────────────────────────────┘
                            │                             │
                            ▼                             ▼
       ┌─────────────────────────────┐      ┌─────────────────────────────┐
       │                             │      │                             │
       │     Monitoring Interface    │      │      Logging Interface      │
       │                             │      │                             │
       │  • Metrics Visualization    │      │  • Log Aggregation          │
       │  • Alerting Dashboards      │      │  • Log Search & Analytics   │
       │  • Performance Monitoring   │      │  • Log Visualization        │
       │  • Resource Utilization     │      │  • Error Tracking           │
       │                             │      │                             │
       └─────────────────────────────┘      └─────────────────────────────┘
                   ▲                                      ▲
                   │                                      │
                   └──────────────────┬───────────────────┘
                                      │
                                      ▼
                        ┌─────────────────────────────┐
                        │                             │
                        │      DevOps Engineer        │
                        │                             │
                        └─────────────────────────────┘

Key Components

1. Infrastructure Layer

   • AWS EC2 instances with appropriate security groups
   • Networking with VPC, subnets, and security groups
   • RKE2 Kubernetes cluster with high-availability configuration

2. Monitoring Stack

   • Prometheus Operator for metrics collection
   • Grafana for visualization with pre-configured dashboards
   • Alertmanager for alert routing and notifications
   • Node exporters for host-level metrics

3. Logging Stack

   • Elasticsearch for log storage and indexing
   • Fluentd for log collection (deployed as DaemonSet)
   • Kibana for log visualization and analysis

4. Security & Access

   • Nginx Ingress Controller for external access
   • Cert-Manager with Let's Encrypt for TLS certificates
   • RBAC for access control within Kubernetes

5. Infrastructure Management

   • Terraform modules for all components
   • Helm charts for application deployment
   • Configuration management through Terraform

All components are deployed as Infrastructure as Code, enabling consistent, repeatable deployments and easy maintenance.

🧰 Components

• Infrastructure: AWS EC2 instances with RKE2 Kubernetes cluster
• Monitoring: Prometheus, Grafana, AlertManager (via Terraform Helm provider)
• Logging: EFK (Elasticsearch, Fluentd, Kibana) (via Terraform Helm provider)
• Security: TLS certificates via cert-manager and Let's Encrypt
• Access: ingress-nginx for external service access

📂 Project Structure

• terraform-rke2/: Terraform module for AWS RKE2 Kubernetes infrastructure
• kube-promstack/: Terraform module for Prometheus monitoring stack
• logging-stack/: Terraform module for EFK logging stack
• scripts/: Utility scripts for deployment and configuration

🚀 Getting Started

Prerequisites

• AWS account with appropriate permissions
• Terraform (>= 0.14.0)
• kubectl (for cluster management)

Deployment

# Clone the repository
git clone https://github.com/Teddydesta/terraform-rke2-monitoring-logging.git
cd terraform-rke2-monitoring-logging

# Deploy infrastructure and Kubernetes cluster
cd terraform-rke2
terraform init
terraform apply

# Deploy Prometheus monitoring stack
cd ../kube-promstack
terraform init
terraform apply -var="kube_config_path=../terraform-rke2/kubeconfig"

# Deploy EFK logging stack
cd ../logging-stack
terraform init
terraform apply -var="kube_config_path=../terraform-rke2/kubeconfig"

# Deploy additional components
cd ..
./scripts/deploy-extras.sh

Using the Makefile

Alternatively, use the provided Makefile for simpler deployment:

# Deploy everything
make deploy-all

# Or deploy individual components
make deploy-infra
make deploy-monitoring
make deploy-logging
make deploy-extras

📋 Detailed Documentation

For more detailed information, refer to the module directories:

• terraform-rke2/: AWS infrastructure and RKE2 Kubernetes cluster configuration
• kube-promstack/: Prometheus monitoring stack setup
• logging-stack/: EFK logging stack configuration

🤝 Contributing

Contributions are welcome! Please create a pull request with your changes.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔮 Future Improvements

While the current implementation provides a comprehensive monitoring and logging solution, several DevOps-focused enhancements could further improve the system:

1. GitOps Workflow Implementation: Adopt Flux or ArgoCD to manage infrastructure and application deployments using Git as the single source of truth, enabling automated drift detection and reconciliation.

2. DevSecOps Pipeline Enhancement: Integrate security scanning (Trivy, Clair) into the CI/CD pipeline with automated vulnerability management and compliance checks. Implement OPA Gatekeeper for policy enforcement.

3. Infrastructure as Code Evolution: Migrate from Terraform HCL to Terraform CDK (or Pulumi) for advanced infrastructure modeling with proper abstractions, unit testing, and type safety.

4. Observability Platform Integration: Build a unified observability platform connecting Prometheus metrics, EFK logs, and distributed tracing (Jaeger/OpenTelemetry) with correlation capabilities.

5. Multi-Environment Promotion Pipeline: Implement a robust promotion pipeline (dev → staging → production) with automated canary deployments and progressive delivery using Flagger and service mesh.

6. Infrastructure Self-Service Portal: Create an internal developer platform using Backstage to enable self-service infrastructure provisioning with guardrails and governance.

7. Chaos Engineering Practice: Integrate Chaos Mesh or Litmus Chaos for automated chaos experiments to validate system resilience, with experiment results feeding into the CI/CD process.

8. FinOps Implementation: Deploy Kubecost or OpenCost with chargeback/showback capabilities, right-sizing recommendations, and automated scaling policies based on cost metrics.

9. Zero-Trust Security Model: Implement service mesh (Istio/Linkerd) with mTLS, network policies, and identity-based access controls across the entire infrastructure stack.

10. Compliance as Code: Integrate tools like Checkov or Terratest to automate compliance checks against CIS Benchmarks, SOC2, and other standards as part of the Infrastructure as Code workflow.

These improvements would transform the platform from a traditional monitoring solution into a comprehensive, enterprise-grade DevOps platform enabling sophisticated delivery workflows, enhanced security posture, and advanced operational capabilities.