feat(api)!: redesign node log retrieval API around tag-based queries

[spalladino]

Motivation

The node exposed four log-retrieval methods with three filter shapes and two return shapes, while the private index was only tag-keyed — so a (tag, narrow block range) query loaded the entire per-tag history into memory. Pagination was a single global page counter shared across all tags, and the public path was split between a LogFilter method and a tag-based method. v5 is already a breaking release, so this collapses everything to a single, fast, tag-based surface with no back-compat.

Fixes A-1111
Fixes A-1031

Approach

Two methods, getPrivateLogsByTags(query) and getPublicLogsByTags(query), replace the four old ones; getContractClassLogs and getPublicLogs(LogFilter) are gone. The archiver stores each log under a fixed-width composite hex-string key [contractHex] - tagHex - blockHex8 - txIndexHex8 - logIndexHex8 in an LMDB map, so every supported filter (tag, block range, txHash, per-tag afterLog cursor, referenceBlock reorg cap) reduces to a single ordered range scan. Note hashes and nullifiers are never copied into the log index — they're fetched on demand from the block store via a partial deserializer that reads only the relevant prefix of the stored IndexedTxEffect. ARCHIVER_DB_VERSION bumps 6 → 7, so the archiver self-wipes and re-syncs from L1 on first start.

API

Two node methods replace the previous four. Each returns one inner array per element of query.tags, in input order; an empty inner array means that tag matched nothing.

getPrivateLogsByTags(query: PrivateLogsQuery): Promise<LogResult[][]>;
getPublicLogsByTags(query: PublicLogsQuery): Promise<LogResult[][]>;

Input

// Filters shared by both queries.
type LogsQueryBase = {
  fromBlock?: BlockNumber;        // inclusive lower bound
  toBlock?: BlockNumber;          // exclusive upper bound
  txHash?: TxHash;                // restrict to one tx; mutually exclusive with fromBlock/toBlock
  referenceBlock?: BlockHash;     // reorg anchor: throws if that block is no longer present
  includeEffects?: boolean;       // also attach each log's tx noteHashes + nullifiers
  limitPerTag?: number;           // page size, 1..MAX_LOGS_PER_TAG (default & max = 20)
};

// A tag to query, optionally resuming strictly after a previously-seen log.
// The bare `T` form starts from the beginning.
type TagQuery<T> = T | { tag: T; afterLog?: LogCursor };

type PrivateLogsQuery = LogsQueryBase & {
  tags: TagQuery<SiloedTag>[];    // 1..MAX_RPC_LEN (100) entries
};

type PublicLogsQuery = LogsQueryBase & {
  contractAddress: AztecAddress;  // required for public queries
  tags: TagQuery<Tag>[];          // 1..MAX_RPC_LEN (100) entries
};

Output

type LogResult<Opts = { includeEffects?: boolean }> = {
  logData: Fr[];                  // log fields; the tag is logData[0]
  blockNumber: BlockNumber;
  blockHash: BlockHash;
  blockTimestamp: UInt64;
  txHash: TxHash;
  txIndexWithinBlock: number;     // 0-based index of the tx within its block
  logIndexWithinTx: number;       // 0-based index of the log within its tx
} & (Opts extends { includeEffects: true }
  ? { noteHashes: Fr[]; nullifiers: Fr[] } // present only when includeEffects: true
  : {});

// Opaque per-tag pagination cursor.
// String form: `<blockNumber>-<txIndexWithinBlock>-<logIndexWithinTx>`.
class LogCursor {
  blockNumber: BlockNumber;
  txIndexWithinBlock: number;
  logIndexWithinTx: number;
  static fromLog(log: LogResult): LogCursor;
}

Pagination is per-tag: feed a tag's last LogResult back as the next query's afterLog ({ tag, afterLog: LogCursor.fromLog(last) }). A tag is exhausted once it returns fewer than limitPerTag results. The stdlib helpers queryAllPrivateLogsByTags / queryAllPublicLogsByTags drive this loop and return the fully-drained results.

Changes

• stdlib: new LogResult, LogCursor, PrivateLogsQuery / PublicLogsQuery types with zod schemas; txHash ⊕ fromBlock/toBlock enforced via .refine (but txHash + afterLog is allowed, to paginate within a tx's logs). L2LogsSource / AztecNode / Archiver interfaces and schemas reduced to the two new methods. Deleted LogFilter, LogId, TxScopedL2Log, ExtendedPublicLog, ExtendedContractClassLog, GetPublicLogsResponse, GetContractClassLogsResponse, and the dead Tx.getPublicLogs(logsSource).
• archiver: full LogStore rewrite — two hex-string-keyed AztecAsyncMap primary maps (keys are fixed-width zero-padded lowercase hex, so ordered-binary's string ordering matches the canonical (contract, tag, block, txIndex, logIndex) tuple and every filter is a single ordered range scan) plus two blockNumber → string[] secondary indices driving deleteLogs (replaces the buggy per-block tag-union list). All reads, including the referenceBlock existence check, run inside one db.transactionAsync across BlockStore + LogStore; a referenceBlock equal to the (synthetic, unindexed) genesis block hash resolves to the genesis block number rather than throwing. New BlockStore.getNoteHashesAndNullifiers(txHashes) is a batched partial deserializer for includeEffects, and getTxLocation reads only the 40-byte header instead of the full TxEffect. Contract-class-log storage removed entirely. ARCHIVER_DB_VERSION 6 → 7. OutOfOrderLogInsertionError and the ARCHIVER_MAX_LOGS env var dropped.
• aztec-node: four RPC handlers collapsed to two thin forwarders; referenceBlock resolution moved into the store so it shares the transaction.
• pxe: getAllPages rewritten from a global page counter to per-tag afterLog cursors — each round re-queries only tags that returned a full page, and tags drop out as soon as they return a short page. fromBlock / toBlock are pushed down into the node, eliminating the in-memory #extractLogs range filter.
• aztec.js: getPublicEvents migrated to the new query shape; PublicEventFilter.contractAddress is now required; EventFilterBase.afterLog: LogId → LogCursor.
• cli: get-logs requires --contract-address and --tag; --after-log parses a LogCursor string <blockNumber>-<txIndexWithinBlock>-<logIndexWithinTx>.
• end-to-end: e2e_ordering rewritten to read getBlock().body.txEffects[*].publicLogs directly (the new API drops the tag-less, contract-less query shape).
• docs: migration notes for client consumers; operator changelog covering the DB version bump and one-time resync.

[PhilWindle]

Node changes look fine