feat: add OTLP logging export

docs/dev/telemetry.md

@@ -52,6 +52,13 @@ Telemetry is configured via environment variables:
 | `MELLEA_METRICS_PROMETHEUS` | Enable Prometheus metric reader (registers with prometheus_client registry) | `false` |
 | `OTEL_METRIC_EXPORT_INTERVAL` | Export interval in milliseconds | `60000` |
 
+#### Logging Configuration
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `MELLEA_LOGS_OTLP` | Enable OTLP logs exporter | `false` |
+| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | OTLP logs-specific endpoint (overrides general endpoint) | None |
+
 ### Application Trace Scope
 
 The application tracer (`mellea.application`) instruments:
@@ -561,6 +568,105 @@ Enable at least one exporter:
 - OTLP: `export MELLEA_METRICS_OTLP=true` + endpoint
 - Prometheus: `export MELLEA_METRICS_PROMETHEUS=true`
 
+### OTLP Logging Export
+
+Mellea's internal logging (via `FancyLogger`) can export logs to OTLP collectors for centralized log aggregation and analysis. This complements the existing tracing and metrics capabilities.
+
+**Note**: OTLP logging is **disabled by default** and requires explicit enablement. When disabled, Mellea's logging works normally (console + REST handlers) with zero overhead.
+
+#### Configuration
+
+Enable OTLP log export with environment variables:
+
+```bash
+# Enable OTLP logs exporter
+export MELLEA_LOGS_OTLP=true
+
+# Configure OTLP endpoint (required)
+export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
+
+# Optional: Use logs-specific endpoint (overrides general endpoint)
+export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://localhost:4318
+
+# Optional: Set service name
+export OTEL_SERVICE_NAME=my-mellea-app
+```
+
+#### How It Works
+
+When enabled, Mellea's `FancyLogger` adds an OpenTelemetry `LoggingHandler` alongside existing handlers:
+- **Console handler**: Continues to work normally
+- **REST handler**: Continues to work normally
+- **OTLP handler**: Exports logs to configured OTLP collector (new)
+
+Logs are exported using OpenTelemetry's Logs API with batched processing for efficiency.
+
+#### OTLP Collector Setup Example
+
+```bash
+# Create otel-collector-config.yaml
+cat > otel-collector-config.yaml <<EOF
+receivers:
+  otlp:
+    protocols:
+      grpc:
+        endpoint: 0.0.0.0:4317
+
+exporters:
+  debug:
+    verbosity: detailed
+  file:
+    path: ./mellea-logs.json
+
+service:
+  pipelines:
+    logs:
+      receivers: [otlp]
+      exporters: [debug, file]
+EOF
+
+# Start OTLP collector
+docker run -p 4317:4317 \
+  -v $(pwd)/otel-collector-config.yaml:/etc/otelcol/config.yaml \
+  -v $(pwd):/logs \
+  otel/opentelemetry-collector:latest
+```
+
+#### Integration with Observability Platforms
+
+OTLP logs work with any OTLP-compatible platform:
+- **Grafana Loki**: Log aggregation and querying
+- **Elasticsearch**: Log storage and analysis
+- **Datadog**: Unified logs, traces, and metrics
+- **New Relic**: Centralized logging
+- **Splunk**: Log analysis and monitoring
+
+#### Performance
+
+- **Zero overhead when disabled**: No OTLP handler created, no performance impact
+- **Batched export**: Logs are batched and exported asynchronously
+- **Non-blocking**: Log export never blocks application code
+- **Minimal overhead when enabled**: OpenTelemetry's efficient batching minimizes impact
+
+#### Troubleshooting
+
+**Logs not appearing:**
+1. Verify `MELLEA_LOGS_OTLP=true` is set
+2. Check that OTLP endpoint is configured and reachable
+3. Verify OTLP collector is running and configured to receive logs
+4. Check collector logs for connection errors
+
+**Warning about missing endpoint:**
+```
+WARNING: OTLP logs exporter is enabled but no endpoint is configured
+```
+Set either `OTEL_EXPORTER_OTLP_ENDPOINT` or `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`.
+
+**Connection refused:**
+1. Verify OTLP collector is running: `docker ps | grep otel`
+2. Check endpoint URL is correct (default: `http://localhost:4317`)
+3. Verify network connectivity: `curl http://localhost:4317`
+
 ### Future Enhancements
 
 Planned improvements to telemetry:
@@ -578,6 +684,5 @@ Planned improvements to telemetry:
 - Integration with LangSmith and other LLM observability tools
 
 **Logging:**
-- Structured logging with OpenTelemetry Logs API
-- Log correlation with traces and metrics
-- Log export to OTLP collectors
+- Log correlation with traces and metrics (automatic trace/span ID injection)
+- Structured log attributes for better querying

mellea/core/utils.py

@@ -124,5 +124,14 @@ def get_logger():
             # stream_handler.setLevel(logging.INFO)
             stream_handler.setFormatter(CustomFormatter(datefmt="%H:%M:%S,%03d"))
             logger.addHandler(stream_handler)
+
+            # Add OTLP handler if enabled
+            from ..telemetry import get_otlp_log_handler
+
+            otlp_handler = get_otlp_log_handler()
+            if otlp_handler:
+                otlp_handler.setFormatter(JsonFormatter())
+                logger.addHandler(otlp_handler)
+
             FancyLogger.logger = logger
         return FancyLogger.logger

mellea/telemetry/__init__.py

@@ -1,14 +1,15 @@
 """OpenTelemetry instrumentation for Mellea.
 
 This package provides observability capabilities for Mellea through OpenTelemetry,
-enabling tracing and metrics collection for both application-level operations and
+enabling tracing, metrics, and logging for both application-level operations and
 backend LLM interactions.
 
 Package Structure:
     - tracing: Distributed tracing with two independent scopes:
         * Application traces (mellea.application): User-facing operations
         * Backend traces (mellea.backend): LLM backend interactions
     - metrics: Metrics collection for counters, histograms, and up-down counters
+    - logging: Log export via OTLP
     - backend_instrumentation: Automatic instrumentation for backend operations
 
 Configuration:
@@ -30,12 +31,19 @@
         - OTEL_METRIC_EXPORT_INTERVAL: Export interval in milliseconds (default: 60000)
         - OTEL_SERVICE_NAME: Service name for metrics (default: mellea)
 
+    Logging:
+        - MELLEA_LOGS_OTLP: Enable OTLP log export (default: false)
+        - OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: Logs-specific endpoint (optional)
+        - OTEL_EXPORTER_OTLP_ENDPOINT: General OTLP endpoint (fallback)
+        - OTEL_SERVICE_NAME: Service name for logs (default: mellea)
+
 Dependencies:
     OpenTelemetry packages are optional. If not installed, telemetry features
     are gracefully disabled. Install with: pip install mellea[telemetry]
 
 Example:
-    from mellea.telemetry import trace_application, create_counter
+    from mellea.telemetry import trace_application, create_counter, get_otlp_log_handler
+    import logging
 
     # Trace application operations
     @trace_application("my_operation")
@@ -45,8 +53,15 @@ def my_function():
     # Collect metrics
     counter = create_counter("mellea.requests", unit="1")
     counter.add(1, {"backend": "ollama"})
+
+    # Export logs via OTLP
+    logger = logging.getLogger("my_app")
+    handler = get_otlp_log_handler()
+    if handler:
+        logger.addHandler(handler)
 """
 
+from .logging import get_otlp_log_handler
 from .metrics import (
     create_counter,
     create_histogram,
@@ -70,6 +85,7 @@ def my_function():
     "create_histogram",
     "create_up_down_counter",
     "end_backend_span",
+    "get_otlp_log_handler",
     "is_application_tracing_enabled",
     "is_backend_tracing_enabled",
     "is_metrics_enabled",

mellea/telemetry/logging.py

@@ -0,0 +1,130 @@
+"""OpenTelemetry logging instrumentation for Mellea.
+
+Provides log export using OpenTelemetry Logs API with OTLP exporter support.
+
+Configuration via environment variables:
+- MELLEA_LOGS_OTLP: Enable OTLP logs exporter (default: false)
+- OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: Logs-specific endpoint (optional, overrides general)
+- OTEL_EXPORTER_OTLP_ENDPOINT: General endpoint for all signals (fallback)
+- OTEL_SERVICE_NAME: Service name for logs (default: mellea)
+
+Example:
+    export MELLEA_LOGS_OTLP=true
+    export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
+
+Programmatic usage:
+    from mellea.telemetry.logging import get_otlp_log_handler
+    import logging
+
+    logger = logging.getLogger("my_logger")
+    handler = get_otlp_log_handler()
+    if handler:
+        logger.addHandler(handler)
+        logger.info("This log will be exported via OTLP")
+"""
+
+import os
+import warnings
+from typing import Any
+
+# Try to import OpenTelemetry, but make it optional
+try:
+    from opentelemetry._logs import set_logger_provider
+    from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
+    from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
+    from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
+    from opentelemetry.sdk.resources import Resource
+
+    _OTEL_AVAILABLE = True
+except ImportError:
+    _OTEL_AVAILABLE = False
+
+# Configuration from environment variables
+_LOGS_OTLP = _OTEL_AVAILABLE and os.getenv("MELLEA_LOGS_OTLP", "false").lower() in (
+    "true",
+    "1",
+    "yes",
+)
+_OTLP_LOGS_ENDPOINT = os.getenv("OTEL_EXPORTER_OTLP_LOGS_ENDPOINT") or os.getenv(
+    "OTEL_EXPORTER_OTLP_ENDPOINT"
+)
+_SERVICE_NAME = os.getenv("OTEL_SERVICE_NAME", "mellea")
+
+
+def _setup_logger_provider() -> Any:
+    """Set up the LoggerProvider with OTLP exporter.
+
+    Returns:
+        LoggerProvider instance or None if OpenTelemetry is not available or no endpoint configured
+    """
+    if not _OTEL_AVAILABLE:
+        return None
+
+    if not _OTLP_LOGS_ENDPOINT:
+        warnings.warn(
+            "OTLP logs exporter is enabled (MELLEA_LOGS_OTLP=true) but no endpoint is configured. "
+            "Set OTEL_EXPORTER_OTLP_LOGS_ENDPOINT or OTEL_EXPORTER_OTLP_ENDPOINT to export logs.",
+            UserWarning,
+            stacklevel=2,
+        )
+        return None
+
+    resource = Resource.create({"service.name": _SERVICE_NAME})  # type: ignore
+    logger_provider = LoggerProvider(resource=resource)  # type: ignore
+
+    try:
+        otlp_exporter = OTLPLogExporter(endpoint=_OTLP_LOGS_ENDPOINT)  # type: ignore
+        logger_provider.add_log_record_processor(
+            BatchLogRecordProcessor(otlp_exporter)  # type: ignore
+        )
+    except Exception as e:
+        warnings.warn(
+            f"Failed to initialize OTLP logs exporter: {e}. "
+            "Logs will not be exported via OTLP.",
+            UserWarning,
+            stacklevel=2,
+        )
+        return None
+
+    set_logger_provider(logger_provider)  # type: ignore
+    return logger_provider
+
+
+# Initialize logger provider if OTLP logging is enabled
+_logger_provider = None
+
+if _LOGS_OTLP:
+    _logger_provider = _setup_logger_provider()
+
+
+def get_otlp_log_handler() -> Any:
+    """Get an OTLP logging handler for Python's logging module.
+
+    Returns:
+        LoggingHandler instance if OTLP logging is enabled and configured,
+        None otherwise.
+
+    Example:
+        import logging
+        from mellea.telemetry.logging import get_otlp_log_handler
+
+        logger = logging.getLogger("my_app")
+        handler = get_otlp_log_handler()
+        if handler:
+            logger.addHandler(handler)
+            logger.info("This log will be exported via OTLP")
+    """
+    if _logger_provider is None:
+        return None
+
+    try:
+        handler = LoggingHandler(logger_provider=_logger_provider)  # type: ignore
+        return handler
+    except Exception as e:
+        warnings.warn(
+            f"Failed to create OTLP logging handler: {e}. "
+            "Logs will not be exported via OTLP.",
+            UserWarning,
+            stacklevel=2,
+        )
+        return None