chore: test to prod

[mayankpande88]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @mayankpande88, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers a major upgrade to the L7 tracing capabilities, particularly focusing on Go TLS and HTTP/2 traffic. The core intent is to provide more reliable and detailed monitoring for modern application protocols, with a significant emphasis on capturing and analyzing LLM API interactions, including streaming scenarios. The changes address several challenges in eBPF-based tracing, such as race conditions in event processing and the complexities of Go runtime and HTTP/2 protocol parsing, leading to a more robust and insightful observability solution.

Highlights

• Enhanced LLM API Monitoring: Introduced comprehensive monitoring for Large Language Model (LLM) API calls, including streaming responses. This involves new data structures for LLM request/response parsing, a dedicated stream tracker for Server-Sent Events (SSE) based completion detection, and detailed Prometheus metrics for requests, token usage, time-to-first-token (TTFT), request duration, tokens per second, and errors.
• Improved Go TLS and HTTP/2 Tracing: Significantly enhanced the reliability and accuracy of Go TLS and HTTP/2 tracing. This includes dynamic discovery of Go TLS offsets using DWARF and ITAB information, proactive tracking of Go TLS connections in eBPF to mitigate goroutine scheduling issues, and a more robust HTTP/2 frame parsing logic that handles partial frames and HPACK decoding errors gracefully.
• Robust eBPF Event Handling: Migrated L7 event processing from per-CPU perf buffers to a global ring buffer, ensuring correct event ordering for streaming protocols. Implemented a retry mechanism for L7 events that arrive before their corresponding TCP connections are fully established, reducing data loss due to race conditions. Direct socket information extraction from file descriptors in eBPF provides a fallback for L7 event processing when TCP connection tracking fails.
• Refactored Container Metrics Collection: Optimized the Collect method in the Container struct by removing verbose timing logs and introducing read/write locks for concurrent map access, improving performance and stability. LLM-specific metrics are now emitted directly at the point of completion rather than being aggregated in the main Collect loop.
• Dynamic Workload Resolution: Updated container workload resolution logic to differentiate between Kubernetes and non-Kubernetes containers, allowing for more accurate workload identification based on container ID patterns.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• containers/container.go
  • Removed unused 'log' import.
  • Added 'Operation' field to LLMStats struct for OTel GenAI operation names.
  • Introduced 'googleHTTP2Parsers' map and 'llmStreamTracker' to the Container struct.
  • Modified NewContainer to resolve workload based on container type (k8s vs. non-k8s) and initialize the LLM stream tracker.
  • Updated Close method to stop the LLM stream tracker gracefully.
  • Refactored Collect method to remove detailed timing logs and add read locks for concurrent map access (failedConnectionAttempts, activeConnections, logParsers, processes).
  • Removed direct LLM metrics collection from Collect, delegating it to the LLM stream completion callback.
  • Added createConnectionFromSocketInfo to create ActiveConnection from eBPF socket info when TCP tracking fails.
  • Modified trackLLMRequest to include path, determine operation, and emit Prometheus metrics directly.
  • Implemented onLLMStreamComplete callback for processing completed LLM streaming responses.
  • Updated onL7Request to use onL7RequestWithResult, which includes socket info and retry logic.
  • Enhanced HTTP/2 processing in onL7RequestWithResult with per-connection HPACK parsers and LLM stream tracking integration.
  • Added processHTTP2WithoutConnection to handle HTTP/2 events when connection tracking is unavailable.
  • Refactored updateDelays to use read/write locks for safe concurrent access to process data.
  • Removed debug logging for SSL uprobes.
  • Updated detectLLMFromHTTPRequest to use a centralized path-based detection.
  • Added detectProviderFromRequestStructure for LLM provider detection from request payload structure.
• containers/http_processor.go
  • Added 'bytes' and 'inet.af/netaddr' imports.
  • Introduced DNSResolver type for dynamic hostname lookups.
  • Extended HTTPRequestContext with IsSSE, llmProvider, and llmProviderDone fields.
  • Refactored NewHTTPRequestContext to accept a DNSResolver and streamline processing steps.
  • Renamed parseHTTPRequest to parseRequest and updated its logic.
  • Implemented a priority-based resolveHost method for accurate hostname determination.
  • Updated encodePayloads to handle binary responses and extract only the body for requests.
  • Enhanced extractTraceID to support W3C Trace Context, B3, and custom trace ID headers.
  • Added detectSSE to identify Server-Sent Events.
  • Implemented GetLLMProvider and IsLLMRequest with caching for efficiency.
  • Added helper functions: stripPort, isIPAddress, extractHTTPBody, extractHTTPHeaders, parseW3CTraceID.
• containers/llm.go
  • Added 'net', 'url', and 'regexp' imports.
  • Updated LLMProvider constants to align with OTel standard naming conventions.
  • Introduced OTel GenAI operation names constants.
  • Added 'Operation' field to LLMRequest struct.
  • Expanded llmProviders map and added regex patterns for AWS Bedrock and Azure OpenAI hostnames.
  • Implemented providerDefaultHost to provide canonical hostnames.
  • Modified DetectLLMProvider to handle port stripping and regex-based provider detection.
  • Added DetectLLMProviderFromPath for path-based LLM provider detection.
  • Introduced helper functions: extractGoogleModelFromPath, extractBedrockModelFromPath, isOpenAICompatiblePath, isExactAPIPath, normalizeAPIPath.
  • Added GetOperation to derive OTel GenAI operation names from request paths.
  • Updated ParseLLMRequest and ParseLLMResponse to accept a path parameter and use new helper functions for JSON extraction and provider-specific parsing.
  • Added extractJSON helper to robustly find JSON in payloads.
  • Implemented parseBedrockRequest and parseBedrockResponse for AWS Bedrock API parsing.
• containers/llm_grpc.go
  • Added new file to provide gRPC-specific LLM helpers.
  • Implemented ExtractGRPCServiceMethod to parse gRPC service and method from paths.
  • Added IsGeminiGRPCService to identify Google Gemini gRPC services.
• containers/llm_metrics.go
  • Added new file to define Prometheus metrics for LLM API usage.
  • Defined CounterVecs for ContainerLLMRequestsTotal, ContainerLLMTokenUsageTotal, ContainerLLMErrorsTotal.
  • Defined HistogramVecs for ContainerLLMTimeToFirstToken, ContainerLLMRequestDuration, ContainerLLMTokensPerSecond.
  • Implemented RegisterLLMMetrics to register all LLM-related metrics.
  • Added RecordLLMStreamMetrics to record metrics for completed LLM streams.
  • Implemented categorizeHTTPError to map HTTP status codes to error types.
• containers/llm_stream.go
  • Added new file to implement LLM streaming request tracking.
  • Defined StreamState enum for tracking stream lifecycle.
  • Introduced LLMStream struct to hold detailed information about an active LLM stream.
  • Implemented LLMStreamTracker to manage active streams, including callbacks, garbage collection, and stream limits.
  • Added methods for stream lifecycle events: OnRequestHeaders, OnResponseHeaders, OnDataFrame, OnStreamEnd.
  • Implemented checkSSECompletion to detect Server-Sent Events completion markers.
  • Added completeStream to finalize stream processing and trigger metric emission.
  • Implemented runGC and evictOldestLocked for periodic cleanup of idle/old streams.
  • Added helper functions: streamKey, parseTraceParent, generateTraceID, generateSpanID.
  • Implemented extractTokensFromBuffer with provider-specific token extraction logic for OpenAI, Anthropic, Gemini, and Bedrock.
• containers/registry.go
  • Added pendingL7Events slice and pendingL7EventsLock to Registry for handling delayed L7 events.
  • Introduced pendingL7Event struct to store queued L7 events.
  • Called RegisterLLMMetrics in NewRegistry to ensure LLM metrics are registered.
  • Removed debug logging for Prometheus un/registration.
  • Modified handleEvents to use processL7Event and processPendingL7Events for robust L7 event handling.
  • Implemented processL7Event to handle L7 events, queueing them for retry if connections are not yet established.
  • Added queueL7EventForRetry and processPendingL7Events to manage the retry queue for L7 events.
  • Removed debug logging for container registration.
  • Added ignoreControlPlane check for systemd containers in calcId to filter out irrelevant workloads.
• ebpftracer/ebpf/ebpf.c
  • Disabled bpf_printk by default, enabling it only when BPF_DEBUG is explicitly defined.
• ebpftracer/ebpf/l7/gotls.c
  • Introduced go_tls_offsets struct and go_tls_offsets_map for dynamic Go TLS offset discovery.
  • Refactored go_crypto_tls_get_fd_from_conn to use dynamic offsets and handle gRPC syscallConn unwrapping.
  • Added ensure_connection_tracked to proactively add Go TLS connections to active_connections map.
  • Added debug logging for Go TLS uprobes.
• ebpftracer/ebpf/l7/http2.c
  • Refactored HTTP/2 frame type and flag definitions for clarity.
  • Added valid_flags_for_frame_type helper for frame flag validation.
  • Renamed is_client_preface to is_http2_client_preface.
  • Implemented parse_http2_frame_header for robust HTTP/2 frame header parsing and validation.
  • Added detect_hpack_static_index to identify request/response HPACK headers.
  • Introduced is_headers_frame_with_hpack for more reliable HEADERS frame detection.
  • Refactored looks_like_http2_frame for stricter HTTP/2 detection, including scanning multiple frames for HEADERS.
  • Added is_likely_http2_port helper for port-based HTTP/2 hints.
• ebpftracer/ebpf/l7/l7.c
  • Added COPY_PAYLOAD_RINGBUF macro for efficient payload copying to ring buffer events.
  • Included socket_info.c for direct socket information extraction.
  • Added socket tuple fields (saddr, daddr, sport, dport, addr_family, socket_info_valid) to l7_event struct.
  • Changed l7_events map from PERF_EVENT_ARRAY to RINGBUF for global event ordering.
  • Implemented send_event, reserve_l7_event, and discard_l7_event for ring buffer management.
  • Modified trace_enter_write to use ensure_connection_tracked for TLS, fallback to socket info for DNS/TLS, and fast-path HTTP/2.
  • Added is_dns_request detection.
  • Updated trace_enter_write and trace_exit_read to use ring buffer event management functions.
  • Modified trace_exit_read to handle DNS responses and fast-path HTTP/2 processing.
• ebpftracer/ebpf/l7/openssl.c
  • Added ensure_connection_tracked calls to OpenSSL write/read uprobes to improve connection tracking.
• ebpftracer/ebpf/socket_info.c
  • Added new eBPF C file for dynamic kernel struct offset discovery using BTF.
  • Defined socket_info_offsets struct and socket_info_offsets_map.
  • Introduced socket_tuple struct for extracted socket information.
  • Implemented get_socket_tuple_from_fd to extract socket info from a file descriptor.
• ebpftracer/ebpf/tcp/state.c
  • Increased MAX_PAYLOAD_SIZE to 8192 bytes to accommodate larger LLM HTTP/2 payloads.
• ebpftracer/go_offsets.go
  • Added new file for user-space Go TLS offset discovery.
  • Defined GoTLSOffsets struct for Go TLS connection field offsets.
  • Introduced GoTLSOffsetsC for C-compatible representation in BPF maps.
  • Included knownGoOffsets for version-based fallback offsets.
  • Implemented DiscoverGoTLSOffsets to find offsets using DWARF debug info or version-based fallbacks.
  • Added helper functions: discoverOffsetsFromDWARF, getMemberOffset, getVersionBasedOffsets.
  • Implemented ToC method for converting GoTLSOffsets to GoTLSOffsetsC.
  • Added DiscoverItabAddresses to find Go interface table addresses for gRPC syscallConn support.
• ebpftracer/l7/clickhouse.go
  • Added panic recovery to ParseClickhouse to handle malformed or incomplete packets gracefully.
• ebpftracer/l7/http2.go
  • Added HTTP/2 flag constants and maxPendingHeaderBlockSize.
  • Extended Http2Request with Authority, ContentType, RequestHeaders, hasResponseStatus, responseEndStream, firstResponseTime, and PartialHeaders fields.
  • Introduced pendingHeaderBlock struct for reassembling fragmented header blocks.
  • Extended Http2Parser with clientPartialFrame, serverPartialFrame, clientPendingHeaders, serverPendingHeaders, clientDecoderDegraded, and serverDecoderDegraded fields.
  • Implemented resetDecoder to create a fresh HPACK decoder after unrecoverable errors.
  • Added ActiveRequestCount and GetActiveStreamsForLLM for monitoring active HTTP/2 streams.
  • Implemented extractHeaderBlockFragment to correctly parse HPACK data from HEADERS frames.
  • Refactored decodeHeaderBlock to handle request/response headers, store all request headers, and manage HPACK decoding errors.
  • Modified Parse method to handle partial frames, reassemble HEADERS/CONTINUATION frames, and track stream completion state.
• ebpftracer/socket_offsets.go
  • Added new file for user-space BTF discovery of kernel socket offsets.
  • Defined SocketInfoOffsets struct mirroring the eBPF struct.
  • Implemented discoverSocketOffsets to find kernel struct offsets using BTF.
  • Added initSocketInfoOffsets to populate the eBPF map with discovered offsets.
  • Introduced extractIPFromSocketInfo, SocketInfo, and GetSocketInfoFromL7Event helpers for socket information handling.
• ebpftracer/tracer.go
  • Increased MaxPayloadSize to 8192 bytes.
  • Added l7EventHeaderSize, tcpEventSize, fileEventSize, and procEventSize constants.
  • Removed EventTypeHTTPFragment and HTTPResponseFragment struct.
  • Added SocketInfo field to Event struct for direct socket information.
  • Introduced ringbufReader to Tracer struct for L7 event processing.
  • Modified Close method to close the ring buffer reader.
  • Called initSocketInfoOffsets in ebpf to initialize socket info discovery.
  • Removed l7_events from perfMaps and added runRingbufEventsReader for it.
  • Refactored runEventsReader to use lostSamplesTracker and parse raw bytes directly for efficiency.
  • Added lostSamplesTracker struct and related functions for improved lost sample logging.
  • Implemented runRingbufEventsReader to process L7 events from the ring buffer, ensuring global ordering.
  • Updated ipPort to accept a byte slice for IP addresses.
• go.mod
  • Updated cloud.google.com/go/compute/metadata from v0.6.0 to v0.9.0.
  • Updated github.com/stretchr/testify from v1.10.0 to v1.11.1.
  • Updated go.opentelemetry.io/otel from v1.34.0 to v1.37.0.
  • Updated go.opentelemetry.io/otel/sdk from v1.34.0 to v1.37.0.
  • Updated go.opentelemetry.io/otel/trace from v1.34.0 to v1.37.0.
  • Updated golang.org/x/net from v0.38.0 to v0.46.0.
  • Updated golang.org/x/sys from v0.36.0 to v0.37.0.
  • Updated golang.org/x/time from v0.8.0 to v0.14.0.
  • Updated various indirect dependencies.
• go.sum
  • Updated checksums for various dependencies, reflecting changes in go.mod.
• main.go
  • Added 'flag' import.
  • Initialized klog flags to prevent duplicate output and set log levels.
• proc/net.go
  • Removed 'net' import.
  • Refactored decodeAddr to use stack-allocated buffers and handle IP byte order more efficiently for /proc/net/tcp parsing.
• tracing/tracing.go
  • Added LLMStreamInfo struct to encapsulate LLM stream details for tracing.
  • Implemented LLMRequest method in Trace to create OpenTelemetry GenAI spans, including attributes for provider, model, operation, token usage, TTFT, and error handling.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request significantly enhances L7 tracing capabilities, especially for Go applications, gRPC, and LLM observability, by introducing features such as ring buffers for L7 events, advanced eBPF techniques for Go TLS and socket info extraction, and new LLM stream tracking with OTel-compliant metrics. However, two security issues require attention: a memory leak in the HTTP/2 parser management and a potential cardinality explosion in Prometheus metrics due to untrusted input. Additionally, there is a minor suggestion regarding some potentially unused code.

[gemini-code-assist]

The googleHTTP2Parsers map stores an Http2Parser for every HTTP/2 connection (keyed by PID and FD). However, there is no logic to remove entries from this map when a connection is closed. This leads to a memory leak that grows with the number of HTTP/2 connections made by processes in the container. Over time, this can lead to memory exhaustion and crash the agent.

To fix this, ensure that entries are removed from googleHTTP2Parsers in the onConnectionClose method or during the garbage collection process.

[gemini-code-assist]

The server_address label in LLM metrics is derived from the untrusted Host or :authority header of the HTTP request. An attacker can exploit this to cause a cardinality explosion by sending many requests with unique Host headers to endpoints that match LLM path patterns (e.g., /v1/chat/completions). This can lead to excessive memory consumption by the Prometheus registry and potentially crash the agent or the monitoring system.

Consider validating the host against a whitelist of known LLM providers or limiting the number of unique values for the server_address label to mitigate this risk.