otlpjsonfile: How to load an export create from otelcol file exporter

[smalenfant]

Component(s)

receiver/otlpjsonfile

Describe the issue you're reporting

I have a file that has been exported using otelcol. I'm not sure exactly how but it starts with the following:

{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "vmc-collectd"
...
      "scopeMetrics": [
        {
          "scope": {
            "name": "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver",
            "version": "0.119.0"
          },
          "metrics": [
            {
              "name": "collectd_processes_disk_octets_read_total",
              "description": "collectd_processes_disk_octets_read_total",
              "sum": {
                "dataPoints": [

The size of each JSON messages inside could go from 1.7MB to 80MB.

I've tried using the following receiver config:

  otlpjsonfile:
    include:
      - /tmp/otel-metrics.json

The application starts with following output:

otel-collector-1  | 2025-06-11T01:11:36.356Z	info	fileconsumer/file.go:267	Started watching file	{"resource": {"service.instance.id": "d8293723-4ba6-4099-a05f-b2982ab34ff2", "service.name": "otelcol-contrib", "service.version": "0.128.0"}, "otelcol.component.id": "otlpjsonfile", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "component": "fileconsumer", "path": "/tmp/otel-metrics.json"}

But there is nothing happening at all. Almost like the file doesn't start reading.

I've got the following service:

      exporters:
        - debug

If I push the first JSON message in Kafka, the kafka receivers picks it up just fine (except that I need to change producer and broken max message size settings).

[github-actions]

Pinging code owners:

• receiver/otlpjsonfile: @atoulme

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

@smalenfant can you please include your full collector config?

[smalenfant]

@axw I actually finally got it reading with this config. I'm not sure why this didn't work in the past. This is working when using the docker image. I can't find where the fingerprint file is at (maybe it doesn't matter).

receivers:
  otlpjsonfile:
    include:
      - /tmp/otel-metrics.json
    start_at: "beginning"

processors:
  batch:


exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    metrics:
      receivers:
        - otlpjsonfile
      exporters:
        - debug

[axw]

@smalenfant thanks! I wonder if that should be the default. The README says:

If a file is updated or added, the receiver will read it in its entirety again.

@atoulme WDYT? Seems like a bug.

[smalenfant]

I've noticed that "if" the file isn't present when starting, it doesn't seem to detect it (when using wildcards).

[smalenfant]

I think I'm running into issue due to the size of the JSON document. Seems like the bigger one (77MB) are not even processed. But there is no output on the Otel collector. The file is called 37.json because it was the 37th JSON document in a larger file.

$ ls -lh 37.json 
-rw-r--r--  1 smalenfa  staff    74M Jun 12 10:44 37.json

Start the collector:

otel-collector  | 2025-06-12T14:44:56.465Z	info	fileconsumer/file.go:267	Started watching file	{"resource": {"service.instance.id": "170e22a5-927e-4e14-9800-71bc5de8a682", "service.name": "otelcol-contrib", "service.version": "0.128.0"}, "otelcol.component.id": "otlpjsonfile", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "component": "fileconsumer", "path": "/tmp/37.json"}
<NO DEBUG>

Usually, I get debug information when reading the file and showing the metrics name for example (using normal verbosity).

I believe it was said the export used a batch size of 5000.

Is there any limitation in the JSON reader? Any "trace" that I would be hitting this limitation?

[smalenfant]

Size seems to be a problem. 1.7MB JSON document is too big too. A 6KB document got processed just fine.