docs(results): document raw vs weighted winner fields#689
Conversation
The aggregator now emits explicit rawWinner/rawWinnerIndex and weightedWinner/weightedWinnerIndex on Compare/Ranking datapoints (rapidata-evaluator). Document them in the result reference: - Describe both winners, their shared index space (Both/Neither as trailing indexes), and the null-on-tie / no-clear-winner behavior. - Mark the generic winner/winner_index as deprecated aliases of the weighted fields (the result file is a plain dict, so this is the deprecation channel) and steer new code to the explicit names. - Clarify how summedUserScores is summed (per-voter aggregated score) and how summedUserScoresRatios normalizes it. - Note the aggregator version floor (4.1.0) so readers know old result files fall back to winner/winner_index. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: lino <68745352+LinoGiger@users.noreply.github.com>
|
Reviewed — this is a docs-only change ( One thing worth double-checking before merge: Possible inconsistency in the Minor/optional:
No bugs, security, or test-coverage concerns — this is text-only and there's nothing to unit test. Nice, focused documentation update. |
Match the evaluator rename (rapidata-evaluator#302): the reliability- weighted winner is winner/winnerIndex, the raw vote-count winner is majorityWinner/majorityWinnerIndex, both null when there is no clear winner. winner_index is documented as a deprecated snake_case alias of winnerIndex. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: lino <68745352+LinoGiger@users.noreply.github.com>
Match the evaluator naming (rapidata-evaluator#302): winner/winnerIndex is the raw vote-count winner, weightedWinner/weightedWinnerIndex is the reliability-weighted winner, both null when there is no clear winner. winner_index stays documented as a deprecated alias of winnerIndex. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: lino <68745352+LinoGiger@users.noreply.github.com>
Companion docs PR to RapidataAI/rapidata-evaluator#302, which adds explicit
rawWinner/rawWinnerIndexandweightedWinner/weightedWinnerIndexto Compare/Ranking results.Why
Results exposed raw counts, weighted user-score sums, and both ratios, but only one generic
winner/winner_index— which follows the weighted result. On close votes the raw-majority and weighted winners can differ, so a consumer could unintentionally read the weighted winner as a raw majority (reported from SDK integration, Compare task).What this PR does (docs only)
Updates
docs/understanding_the_results.md:rawWinner/rawWinnerIndexandweightedWinner/weightedWinnerIndex— both winners, their shared index space (Both/Neitheras trailing indexes), and thenull-on-tie / no-clear-winner behavior.winner/winner_indexas deprecated aliases of the weighted fields. Because the result payload is a plain JSON dict there's no type-level deprecation hook, so docs are the deprecation channel (runtime typed accessors / warnings are the sibling stabilize result schema + typed accessors effort's job, reading these same field names).summedUserScoresis summed (per-voter aggregateduserScore) and howsummedUserScoresRatiosnormalizes it.winner/winner_index.No SDK code changes here — the canonical fields are produced server-side by the evaluator (design decision: evaluator is the single source of truth).
🔗 Session: https://node-39981d64.poseidon.rapidata.internal/