⚙️ Consolidate configuration management architecture

[ivyleavedtoadflax]

Problem

Configuration management is currently inconsistent and fragmented across the codebase:

Current Issues:

1. Mixed configuration approaches: Both old ConfigParser direct usage and new ConfigManager class coexist
2. Global state pollution: Configuration is accessed through global variables and singletons
3. No environment support: Limited support for different environments (dev, staging, prod)
4. Missing validation: Configuration values aren't validated at startup
5. Thread safety concerns: Global config manager may have concurrency issues
6. Hardcoded paths: Config path logic is scattered across multiple modules

Code Examples:

# Old style (still present)
config = configparser.ConfigParser()
config.read(CONFIG_PATH)

# New style  
config_manager = ConfigManager()
instance_name = config_manager.get_instance_name()

Impact

• Inconsistent behavior: Different parts of the application may read config differently
• Maintenance burden: Changes require updates in multiple places
• Testing difficulty: Hard to override config for different test scenarios
• Environment deployment: Cannot easily deploy to different environments
• Error proneness: No validation means silent failures with bad config

Proposed Solution

Phase 1: Consolidate to ConfigManager

• Remove all direct ConfigParser usage
• Standardize on ConfigManager class across all modules
• Create centralized configuration loading and validation
• Add proper error handling for missing/invalid config

Phase 2: Add Environment Support

• Add support for environment variables (e.g., REMOTE_PY_INSTANCE_NAME)
• Implement configuration file discovery (~/.config/remote.py/config.{env}.ini)
• Add command-line config overrides
• Support multiple configuration sources with precedence rules

Phase 3: Validation and Schema

• Implement configuration validation using Pydantic or similar
• Add configuration schema documentation
• Validate configuration at startup with clear error messages
• Add configuration migration support for breaking changes

Phase 4: Advanced Features

• Add configuration caching and invalidation
• Implement configuration watching for live updates
• Add configuration backup and versioning
• Create configuration management CLI commands

Proposed Architecture

from typing import Optional
from pydantic import BaseModel, Field
from pathlib import Path

class RemotePyConfig(BaseModel):
    \"\"\"Configuration schema with validation.\"\"\"
    instance_name: Optional[str] = Field(None, description="Default EC2 instance name")
    region: str = Field("us-east-1", description="Default AWS region")
    config_file: Path = Field(description="Path to configuration file")
    
    class Config:
        env_prefix = "REMOTE_PY_"

class ConfigurationManager:
    \"\"\"Centralized configuration management.\"\"\"
    
    def __init__(self, env: str = "default"):
        self.env = env
        self._config: Optional[RemotePyConfig] = None
    
    @property
    def config(self) -> RemotePyConfig:
        if self._config is None:
            self._config = self._load_config()
        return self._config
    
    def _load_config(self) -> RemotePyConfig:
        # Load from multiple sources with precedence:
        # 1. Environment variables
        # 2. Config file
        # 3. Defaults
        pass

# Usage throughout the codebase
config = ConfigurationManager()
instance_name = config.config.instance_name

Acceptance Criteria

• All configuration access goes through a single interface
• Support for environment variables and multiple config files
• Configuration validation with clear error messages
• Backward compatibility with existing config files
• Thread-safe configuration access
• Comprehensive test coverage for all configuration scenarios
• Documentation for configuration options and precedence rules

Priority

🟡 Medium-High - Important for maintainability and deployment flexibility

Dependencies

• Should be done after critical bug fixes (🐛 Remove duplicate function definitions in instance.py #49)
• May require coordination with CLI improvement work