Skip to content

receiver/awsxray not allowing id to be missing in segment/cause/exception #37414

@Hampus-postkod

Description

@Hampus-postkod

Component(s)

receiver/awsxray

What happened?

Description

There is a bug in receiver/awsxray that make the otel collector crash when receiving a trace containing a segment/cause/exception without an id attribute.

Sending the following trace:

{
  "Id": "1-678fbf94-446a5c2a65badb2f6b1a16c2",
  "Duration": 0.213,
  "LimitExceeded": false,
  "Segments": [
    {
      "Id": "1de48a95158d36ca",
      "Document": {
        "id": "1de48a95158d36ca",
        "name": "api_authorizer",
        "start_time": 1737473940.3929505,
        "trace_id": "1-678fbf94-446a5c2a65badb2f6b1a16c2",
        "end_time": 1737473940.601357,
        "parent_id": "35763592299be764",
        "error": true,
        "aws": {
          "account_id": "465143222094",
          "function_arn": "arn:aws:lambda:eu-west-1:465143222094:function:api_authorizer",
          "cloudwatch_logs": [
            {
              "log_group": "/aws/lambda/api_authorizer"
            }
          ],
          "resource_names": [
            "api_authorizer"
          ]
        },
        "origin": "AWS::Lambda::Function",
        "cause": {
          "working_directory": "/var/task",
          "paths": [
            "/src/authorizer/AuthorizerLambda.ts",
            "/src/authorizer/index.ts",
            "/node_modules/@middy/core/index.js"
          ],
          "exceptions": [
            {
              "message": "Unauthorized",
              "type": "UnauthorizedError",
              "stack": [
                {
                  "path": "/src/authorizer/AuthorizerLambda.ts",
                  "line": 34,
                  "label": "qs.handler"
                },
                {
                  "path": "/src/authorizer/index.ts",
                  "line": 199,
                  "label": "lambdaHandler"
                },
                {
                  "path": "/node_modules/@middy/core/index.js",
                  "line": 121,
                  "label": "yXe"
                }
              ]
            }
          ]
        }
      }
    }
  ]
}

Result in the following error:

1737537832989,panic: runtime error: invalid memory address or nil pointer dereference
1737537832989,[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedbdb8]
1737537832989,goroutine 53 [running]:
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.addCause(0x40010bdcc8?, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/cause.go:61 +0x398
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.populateSpan(0x40010bdcc8, 0x40012a05c0, 0x0, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:172 +0x320
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.segToSpans({0x40012a05b0, 0x40012a05a0, 0x40010979a8, 0x0, 0x40012a0610, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}, ...)"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:80 +0x11c
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.ToTraces({0x40012a4000, 0xabe, 0xabe}, {0xffff40f62e40, 0x245f520})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:68 +0x4c0
1737537832989,github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).start(0x4000206a80)
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/receiver.go:110 +0x114
1737537832989,created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).Start in goroutine 1
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/receiver.go:86 +0x78

Due to that the code in https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awsxrayreceiver/internal/translator/cause.go consider the exception id to be mandatory:

	case awsxray.CauseTypeObject:
		evts := span.Events()
		// not sure whether there are existing events, so
		// append new empty events instead
		exceptionEventStartIndex := evts.Len()
		evts.EnsureCapacity(exceptionEventStartIndex + len(seg.Cause.Exceptions))

		for _, excp := range seg.Cause.Exceptions {
			evt := evts.AppendEmpty()
			evt.SetName(ExceptionEventName)
			attrs := evt.Attributes()
			attrs.EnsureCapacity(8)

			// ID is a required field
			attrs.PutStr(awsxray.AWSXrayExceptionIDAttribute, *excp.ID)
			addString(excp.Message, conventions.AttributeExceptionMessage, attrs)
			addString(excp.Type, conventions.AttributeExceptionType, attrs)
			addBool(excp.Remote, awsxray.AWSXrayExceptionRemoteAttribute, attrs)
			addInt64(excp.Truncated, awsxray.AWSXrayExceptionTruncatedAttribute, attrs)
			addInt64(excp.Skipped, awsxray.AWSXrayExceptionSkippedAttribute, attrs)
			addString(excp.Cause, awsxray.AWSXrayExceptionCauseAttribute, attrs)

			if len(excp.Stack) > 0 {
				stackTrace := convertStackFramesToStackTraceStr(excp)
				attrs.PutStr(conventions.AttributeExceptionStacktrace, stackTrace)
			}
		}
	}

Even though it is not according to AWS documentation

Steps to Reproduce

Send a trace with segment/casuse/exception without id attribute to otel-collector with reciever/awsxray.

Expected Result

Should accept request and keep running

Actual Result

Crashes with null pointer exception.

Collector version

v0.117.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  awsxray:
    transport: udp
    endpoint: 0.0.0.0:2000

processors:
  batch:

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers:
        - awsxray
      processors:
        - batch
      exporters:
        - debug

Log output

1737537832989,panic: runtime error: invalid memory address or nil pointer dereference
1737537832989,[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedbdb8]
1737537832989,goroutine 53 [running]:
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.addCause(0x40010bdcc8?, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/cause.go:61 +0x398
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.populateSpan(0x40010bdcc8, 0x40012a05c0, 0x0, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:172 +0x320
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.segToSpans({0x40012a05b0, 0x40012a05a0, 0x40010979a8, 0x0, 0x40012a0610, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}, ...)"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:80 +0x11c
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.ToTraces({0x40012a4000, 0xabe, 0xabe}, {0xffff40f62e40, 0x245f520})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/internal/translator/translator.go:68 +0x4c0
1737537832989,github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).start(0x4000206a80)
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/receiver.go:110 +0x114
1737537832989,created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).Start in goroutine 1
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver@v0.117.0/receiver.go:86 +0x78

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions