perf(value/codec): prune the codec walk to codec-bearing paths via a StripToCodec preprocessor

[amit-evron-cyera]

Follow-up to #1625, taking up the localized per-stage optimization you said you're open to.

Why this is in scope now (not pre-empting the 2.x pipeline work):

• You're making DecodeUnsafe/EncodeUnsafe the primary codec API in 2.x — so optimizing that specific step is directly on that path, and 2.x will want it regardless.
• It's a localized optimization of one stage (the codec walk), which you said you'd consider — same spirit as a faster Convert helping any pipeline.
• It can't be done in user space (below), so it's not something users can opt into themselves.
• It's self-contained and stateless, and can be gated behind a Settings flag (mirroring unionPrioritySort) if you'd prefer it opt-in.

Idea

DecodeUnsafe/EncodeUnsafe walk the whole value, including subtrees that statically contain no codec. "Contains a codec" is a property of the schema, so resolve it once per call: StripToCodec(type) returns the schema reduced to its codec-bearing paths — Object/Array/Record children with no codec are dropped; Union/Intersect/Tuple/Ref/Cyclic are kept intact. The existing walk runs unchanged on the stripped schema and touches only codec paths.

• Once-per-call type preprocessor — same pattern and placement as UnionPrioritySort.
• No retained state, no caching; the walkers are untouched.
• The input schema is not mutated (verified by deep-freezing the schema and round-tripping) — preserves the Value.Decode/Encode/Clean mutate Union's member #1620 contract.

Why it can't be user-side

Pruning in user space means re-implementing the per-type codec traversal (from_object/from_array/from_union/…), and the 2.x Pipeline API doesn't close the gap — composing stages still calls the full-tree walk. So for codec users it's binary: either DecodeUnsafe prunes, or they pay the full-tree walk.

Numbers

Schema with ~2 codec leaves in a large object (many Union([X, Null]) fields + nested), decoded over a 10k-element array (Node 22): Check + DecodeUnsafe ~260 ms → ~8 ms, byte-identical.

Correctness

• Full runtime suite passes (4434).
• Byte-identical to Value.Decode across the existing codec suite + added cases (extra keys, optional-with-default, wrong-type, nested object/array/union/tuple of codecs).
• Schema-immutability verified via deep-freeze round-trip.

If you'd like it opt-in, I'll add a Settings flag mirroring unionPrioritySort.

[amit-evron-cyera]

Hey @sinclairzx81, this PR is a follow-up from this comment. Did you happen to take a look by any chance?

Best regards

[sinclairzx81]

Hi @amit-evron-cyera, sorry for the delay (I had been quite focused on the 1.3.0 release), but I wanted to get to this sooner.

Did you happen to take a look by any chance?

Yes, I have looked at the PR, but unfortunately I won't be able to merge this one in, at least not in this form. I will explain a bit why below, as this functionality has broader crossover with TypeBox's future design and optimization story moving towards 2.x, but won't be able to accept as of today.

Reasoning

The main reason I'm hesitant to merge this one is mostly due to "implicit" schema pre-processing and transformation being an opaque operation the end user would have no control over, so if things go wrong, the user has no way to work around it. This also applies to the current UnionPrioritySort, which is configurable for now (to remain non-breaking), but will be removed in future versions of TypeBox.

Provisions have already been made to make priority sorting an API of Type.* (available in 1.3.0).

Type.Priority Example

import * as Type from 'typebox'

const T = Type.Union([Type.Priority([   // Preprocessing is explicit and optional
  Type.Script(`{ x: number, y: number }`),
  Type.Script(`{ x: number }`),
  Type.Script(`{ x: number, y: number, z: number }`),
])])
                                        // const T: Type.TUnion<[[Type.TObject<{
                                        //   x: Type.TNumber;
                                        //   y: Type.TNumber;
                                        //   z: Type.TNumber;
                                        // }>, Type.TObject<{
                                        //   x: Type.TNumber;
                                        //   y: Type.TNumber;
                                        // }>, Type.TObject<{
                                        //   x: Type.TNumber;
                                        // }>]]>

import * as Value from 'typebox/value'

const A = Value.Clean(T, {})            // Implicit UnionPrioritySort will be removed
                                        // from Clean, Decode, and Encode in future
                                        // versions

Other optimizers will need to be moved outside of the Value.* operations and organized accordingly.

StripToCodec

In terms of StripToCodec, I do think there is value in exploring optimizing operations like StripToCodec, but in terms of design, the function really needs to be an external call rather than an implicit operation of Encode/Decode. From an implementation standpoint, the following is the general thinking.

// --------------------------------------------------------------------
// User Land (Passed to Framework)
// --------------------------------------------------------------------

const T = Type.Object({ .... }) // User-defined schema with embedded codec

// --------------------------------------------------------------------
// Framework Level (Customizable Optimizations)
// --------------------------------------------------------------------

import { Decode, Encode } from 'typebox/value'
import { StripToCodec } from './optimizers/strip-to-codec.ts'

export function FrameworkDecoder(type: TSchema)  {
  const stripped = StripToCodec({}, type)    // Explicit call before decode
  return (value: unknown) => Decode({}, stripped, value)
}
export function FrameworkEncoder(type: TSchema)  {
  const stripped = StripToCodec({}, type)    // Explicit call before encode
  return (value: unknown) => Encode({}, stripped, value)
}

The above works with the general direction of TypeBox; however, the problem with incorporating StripToCodec as a public API is that the function would need to exist as part of a larger suite of possible optimizers (and be managed as one of potentially many).

Accommodating a singular StripToCodec function really isn't possible today, as TypeBox has no concept of a schema optimizer. That said, it's absolutely something I would like to explore in the future, but it's not clear to me at this stage what optimizers could exist, and what parts of TypeBox would need to be changed in order to accommodate a broader range of them. This would take some design work and planning, and at this stage, there isn't much bandwidth available to explore the concept in depth.

Considerations

Based on the implementation provided in this PR, it does seem like StripToCodec can be moved outside of Encode/Decode and called before these functions, so I think for now it's going to be better to hold off on the PR, but as mentioned, I am happy to start a discussion on the StripToCodec optimization itself.

I'll leave this PR open for now for follow-up discussions on the optimization, but will likely close it out in a few days. I'm very happy to field any follow up discussions on this one.

Cheers,
S