protorev

protorev is a small protobuf reverse-engineering workbench.

The goal is not to replace protoc --decode_raw. The goal is to keep the evidence you need while reverse engineering unknown protobuf streams: offsets, wire types, recursive length-delimited candidates, corpus-level field presence, and a draft schema that stays honest about what was observed.

Install

Install the latest GitHub release with the cargo-dist installer:

curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/patrickomatic/protorev/releases/latest/download/protorev-installer.sh | sh

Or install from crates.io:

cargo install protorev

Prebuilt archives and a PowerShell installer for Windows are available on the GitHub releases page.

Commands

protorev dump sample.pb
protorev dump --json sample.pb
protorev infer samples/*.pb
protorev schema samples/*.pb
protorev explain --field 3.1 samples/*.pb
protorev values --field 1 samples/*.pb
protorev diff before/*.pb -- after/*.pb
protorev experiments experiments.protorev

Input Format

CLI commands read raw protobuf wire payloads from files. They do not read hex text, JSON, .proto schemas, gRPC length-prefixed frames, protobuf-delimited record streams, or application/container formats directly.

If your capture has framing or an outer container, strip that layer first and write each raw protobuf message payload to its own file. For inspectable sample fixtures, keep the hex source in git and materialize bytes before passing them to protorev.

dump

dump decodes one raw protobuf message and prints each field with byte offsets:

@0..3 field 1 varint = 150
@3..10 field 2 length-delimited len=5 [utf8] bytes=74 69 74 6c 65
  text "title"

For length-delimited fields, protorev reports any candidates that match:

• message: the payload decodes cleanly as a nested protobuf message
• utf8: the payload is valid UTF-8 text without non-text control characters
• packed-varint: the payload decodes cleanly as two or more varints

These are candidates, not schema facts. A short byte string can legitimately look like more than one thing.

Use --json when another tool needs offsets, raw values, and hints:

protorev dump --json sample.pb

infer

infer reads multiple samples, aggregates field presence, marks fields as repeated when a field number appears more than once in a sample, recursively tracks nested message candidates, and emits a draft .proto:

samples: 2

root:
  field 1: observed 2/2 samples; wires: length-delimited; max/sample: 1

--- draft proto ---

syntax = "proto3";

message Message {
  Message_1 field_1 = 1; // observed 2/2 samples; wires: length-delimited
}

explain

explain reports the evidence behind one field path:

protorev explain --field 3.1 samples/*.pb

The output names the synthetic field, schema type, confidence, observation counts, wire types, length-delimited candidates, and whether the field would be included at high/medium/low schema thresholds. Use dotted paths for nested message candidates.

For tooling, add --json:

protorev explain --json --field 3.1 samples/*.pb

values

values summarizes observed values for one field path:

protorev values --field 1 samples/*.pb

For numeric fields it reports min, max, distinct count, common values, and conservative candidates such as bool, enum-like, or counter/id-like. For length-delimited fields it reports length ranges plus UTF-8, nested-message, and packed-varint observation counts.

For tooling:

protorev values --json --field 1 samples/*.pb

diff

diff compares two corpora and reports structural changes:

protorev diff before/*.pb -- after/*.pb

It reports added, removed, and changed fields. A changed field can include presence, repetition, wire type, confidence, or length-delimited evidence changes.

The old two-file shorthand still works:

protorev diff before.pb after.pb

For tooling:

protorev diff --json before/*.pb -- after/*.pb

experiments

experiments runs named before/after corpus comparisons from a small manifest:

protorev experiments experiments.protorev

Manifest paths are resolved relative to the manifest file:

[[experiment]]
name = "add title field"
notes = "same message source with only the title populated"
before = ["before/empty.pb"]
after = ["after/with-title.pb"]

[[experiment]]
name = "repeat field 7"
before = ["single.pb"]
after = ["repeated-a.pb", "repeated-b.pb"]

Each experiment prints its name, notes, sample paths, and the same structural diff produced by diff. Use JSON for stable artifacts:

protorev experiments --json experiments.protorev

Manifest syntax is intentionally small:

• experiments start with [[experiment]]
• supported keys are name, notes, before, and after
• name is required; notes is optional
• before and after are required string arrays with at least one path
• relative sample paths are resolved relative to the manifest file
• comments start with # outside quoted strings
• quoted strings support \", \\, \n, \r, and \t
• trailing commas are not supported

schema

schema emits a confidence-gated structural .proto. By default it includes only high-confidence fields:

protorev schema samples/*.pb

High confidence currently means:

• at least two relevant samples were observed
• the field used one stable wire type
• the field appeared in every relevant sample
• if emitted as a nested message, every observed occurrence decoded cleanly as that nested message shape

For exploratory output, lower the threshold:

protorev schema --min-confidence medium samples/*.pb

The emitted schema is intentionally structural:

syntax = "proto3";

message Message {
  Message_1 field_1 = 1; // confidence: high; observed 2/2 samples; wires: length-delimited; occurrences: 2; nested: 2/2; utf8: 0/2; packed-varint: 2/2
}

schema still does not invent semantic names or scalar intent. A stable varint is uint64; a stable length-delimited field is either a consistently observed nested message or bytes.

Suggested Workflow

Start with small, controlled corpora. Capture a baseline message set, change one producer behavior, capture the after set, then compare structure:

protorev diff before/*.pb -- after/*.pb
protorev explain --field 3.1 before/*.pb after/*.pb
protorev values --field 3.1 before/*.pb after/*.pb

Use schema after the same field shape survives several controlled samples. Keep the default high-confidence threshold for durable output; lower it only when exploring hypotheses:

protorev schema before/*.pb after/*.pb
protorev schema --min-confidence medium before/*.pb after/*.pb

Scope

The first version handles raw protobuf wire streams:

• strict decoding for wire types 0, 1, 2, and 5
• byte offsets for every decoded field
• recursive inspection of length-delimited values that parse cleanly as messages
• UTF-8 and packed-varint hints for length-delimited values
• corpus aggregation with draft .proto output

It intentionally does not infer semantic scalar types yet. Varints are emitted as uint64, fixed-width values as fixed32/fixed64, and opaque length-delimited fields as bytes.

Library API

The binary is a thin wrapper around the library:

use protorev::{Corpus, Message, dump_message};

fn main() -> Result<(), protorev::Error> {
    let message = Message::decode(&[0x08, 0x96, 0x01])?;
    println!("{}", dump_message(&message, 4));

    let corpus = Corpus::from_messages(&[message], 4);
    println!("{}", corpus.draft_proto());
    println!("{}", corpus.schema(&Default::default()));
    Ok(())
}

Use Message::decode when you already have raw protobuf bytes. Format-specific adapters should decode their container/framing first, then pass the raw protobuf payload into this crate.

Reverse-Engineering Discipline

protorev reports evidence. It does not claim that:

• a clean nested-message parse proves a field is definitely a message
• printable bytes prove a field is semantically a string
• a varint's numeric value explains what the field means

Promote a hypothesis only after it survives comparison across controlled samples. That is the line between structural knowledge and content coincidence.

Adapters for framed or containerized formats should decode their own outer framing and pass raw protobuf payloads into this crate.