alphaexpr

Parse, check, and de-duplicate formulaic alpha expressions. No dependencies, Python 3.9+.

import alphaexpr as ax

ax.check("group_rank(ts_mean(close, 20), sector)").valid   # True
ax.check("group_rank(close, 20)").errors
# ['group_rank(): argument 1 is a value, not a classifier -- ...']

gate = ax.NoveltyGate(book=my_accepted_alphas, max_similarity=0.6)
gate.check("ts_rank(close, 60)")     # -> too similar (similarity 1.00 vs alpha-014)

---

Why this exists

Published work on formulaic alpha mining — genetic programming, RL, LLM search — scores a candidate on its own: information coefficient, Sharpe, some backtest statistic. Deployed alpha search has a second constraint that the isolated view cannot express:

A candidate must be different from the alphas you have already accepted.

That constraint is not a detail. It is adversarial to your own success: every alpha you accept enlarges the book, which shrinks the region that the next candidate has to land in. A search that ignores it converges on restatements of its own best result and then quietly stops producing anything.

alphaexpr provides the cheap half of that check — structural overlap computed from the parse tree, at zero evaluation cost — plus the syntax and type checks that stop a doomed expression before it consumes a slot in a rate-limited evaluation queue.

It is a pre-filter, not a substitute for correlating realized returns. Two expressions that share no subtrees can still hold the same positions.

Install

pip install git+https://github.com/loversky02/alphaexpr.git

What it does

Parse. A recursive-descent parser for the formula dialect these systems share — nested calls, keyword arguments, arithmetic, comparisons, string literals. Trees are plain tuples: hashable, cheap to copy, easy to mutate in a search loop. unparse round-trips them back to source with only the parentheses that are needed.

ax.parse("ts_decay_linear(close, lookback=20)")
# ('call', 'ts_decay_linear', [('field', 'close'), ('kw', 'lookback', ('num', '20'))])

Measure novelty. Similarity is the Jaccard index over each expression's bag of canonical subtrees. Two choices matter, and both cut against naive shape matching:

• Leaves keep their field identity. Numeric literals collapse, so ts_rank(x, 20) and ts_rank(x, 60) merge — a window is a hyperparameter, not an idea — and so do window numbers inside a field name, wherever they sit: adv20/adv60, and beta_last_30_days_spy/beta_last_360_days_spy. A leading dataset prefix is the exception and survives, so fnd6_acdo and fnd28_acdo stay different. The same skeleton over a different input scores zero and passes. See the empirical section for why this is the most important property here.
• Similarity is measured against the accepted book only, never against every candidate ever evaluated. Variants are supposed to resemble the seed they came from; that is what makes them variants.

ax.similarity("ts_rank(close, 20)", "ts_rank(close, 60)")            # 1.0
ax.similarity("ts_rank(close, 20)", "ts_rank(news_sentiment, 20)")   # 0.0

A subtree index means check only compares against book entries that share at least one subtree, so a candidate built on a new input skips the book almost entirely. At a few hundred entries the cost is dominated by parsing the candidate anyway; the index starts to matter past that.

The gate is mutable on purpose, so you can watch saturation happen instead of discovering it months later:

gate = ax.NoveltyGate(book=accepted, max_similarity=0.6)
for candidate in search():
    if gate.check(candidate):
        if evaluate(candidate).passes:
            gate.add(candidate)          # the target just moved

Check. Syntax and vocabulary (validate), operator/argument compatibility (typecheck), or both at once (check). The type checker targets the three failure classes that dominate rejected evaluations: a group operator handed something that is not a classifier, a time-series operator applied to a sparse event field, and plain arity errors.

Bound size. ComplexityBudget caps operator count and nesting depth. Search procedures drift toward bloat because a bigger tree can always squeeze out a little more in-sample fit; bloat overfits and concentrates weight, which is what portfolio constraints actually reject.

Repair. fill_missing_args deterministically appends arguments that a generator forgot. Faster and more reliable than asking the generator again.

Command line.

alphaexpr check "group_rank(close, 20)"
alphaexpr similarity "ts_rank(close, 20)" "ts_rank(close, 60)"
alphaexpr novelty book.txt candidates.txt --max-similarity 0.6

Dialects

Everything platform-specific lives in a Vocabulary — operators with their arity and kind, classifier tokens, event fields, known-good and known-bad field names. DEFAULT_VOCABULARY follows the naming convention used by WorldQuant's FASTEXPR, which most published formulaic-alpha work borrows. No platform's field catalogue is reproduced here; field checking stays off until you supply one.

vocab = ax.DEFAULT_VOCABULARY.extend(
    fields=my_catalogue,
    event_fields=my_sparse_fields,
    bad_fields=known_nonexistent,
)
vocab.save("dialect.json")           # and Vocabulary.load(...) to restore

For a different dialect entirely, build a Vocabulary from scratch — the parser and the novelty gate do not care about operator names.

---

What ~10,000 evaluations taught

The design above is not from first principles. It comes from operating one automated alpha search against a live platform for several months. These are aggregate observations from a single account in a single market: evidence, not proof, and quite possibly specific to that setting. They are worth writing down anyway, because none of them are visible in the isolated-scoring setup that the literature uses.

1. Passing the quality bar is not the same as being accepted, and the difference is nearly everything.

Three generation strategies, over the account's full history:

strategy	evaluations	cleared the quality bar	accepted into the book
blend two different validated families	274	59	18
perturb one alpha's parameters	372	55	0
perturb one alpha's operators	76	7	0

448 evaluations of perturbation produced 62 alphas that the platform called good and not one that was novel enough to accept. Perturbation moves along the axis the correlation check is most sensitive to. Blending two independently validated families is novel by construction, and it produced every accepted alpha the search ever made.

If your search optimizes a score that looks like the middle column, it will report progress while shipping nothing. Optimize the right column.

2. A new input beats a new structure, and it is not close.

The clearest single result in the whole history: an expression of one field and one operator over a dataset the book had never touched scored 0.31 on the platform's self-correlation check — comfortably clear. Elaborate mined structures built on fields the book already used scored 0.72 to 0.97 and were blocked, including ones with better standalone statistics.

Self-correlation measures return overlap, and returns follow the input. Rearranging operators over the same data mostly rearranges the same signal.

This is why the novelty gate collapses windows but protects dataset identity. A gate that penalised familiar shapes would have suppressed exactly these results; so would one that read the 6 in fnd6_acdo as a window and merged it with fnd28_acdo. Those two gates are one regex apart, and the difference does not show up until it has cost you the only candidates that work.

3. Saturation is self-inflicted and it accelerates.

Every accepted alpha makes the next one harder. Over ten days the internal allocator cut the mining channel's share of generation from 15.1% to 0.7% — not because mining got worse, but because its output kept colliding with a book that its own successes had grown. A backlog of 344 candidates that had already cleared the quality bar was screened later: 81% were blocked as derivatives of the book.

The three that were not derivatives all came from fields the book did not use. See observation 2.

4. Most alphas that die, die on portfolio construction, not on signal.

77% of candidates that cleared the return and risk-adjusted-return thresholds were then rejected for weight concentration or thin sub-universe performance. Standalone predictive power was not the binding constraint; how the position sizes came out was. A search scored purely on IC never sees this failure at all.

5. A checker with false positives is worse than one with gaps.

An earlier validator wrongly rejected roughly one valid expression in ten. That does not cost ten percent of throughput. It removes the unusual candidates first — the ones a saturated search depends on — while the familiar ones sail through. Every check here is built so an error is raised only when rejection is near-certain; anything computed, wrapped, or merely unfamiliar passes and is left for the platform to judge.

The same reasoning drives NoveltyGate.filter(never_empty=True): a gate that empties every batch stops the search, and a stopped search never finds the new input that would have escaped the book. Restored candidates are reported separately so the condition stays visible instead of being papered over.

Limitations

• Structural similarity is a proxy. It cannot see that two different formulas produce the same positions. Correlate realized returns before you rely on a novelty claim.
• The parser covers the common dialect. Platform-specific extensions — vector-field indexing, multi-statement expressions — are not modelled.
• Field normalization assumes a dataset prefix looks like <letters><digits>_ and that every other number is a window. If your catalogue numbers fields by meaning instead, pass strip_windows=False and compare names literally.
• Thresholds (max_similarity=0.6, 16 operators, depth 8) are fitted to one account. Re-fit them against your own accepted set.
• The empirical section is n=1: one account, one market, one search design.

Related work

Formulaic alpha mining is an active field. Where diversity pressure appears at all, it is applied against the search's own history rather than against an external, already-accepted book that grows as the search succeeds — which is the gap this library addresses:

• AlphaForge — generative-predictive architecture for mining and dynamically combining factors
• QuantFactor REINFORCE — variance-bounded REINFORCE for formulaic factor discovery
• Alpha Jungle — LLM-powered MCTS over the formula space. The closest prior art here: its frequent-subtree-avoidance mechanism applies structural diversity pressure over the parse tree, though against the search's own history rather than an accepted book
• Chain-of-Alpha — LLM chains for alpha factor mining
• AlphaBench — benchmarking LLMs on alpha factor mining

License

MIT.