Skip to content

Add average rank computation column#2351

Merged
Didayolo merged 4 commits into
developfrom
average-rank
Jun 18, 2026
Merged

Add average rank computation column#2351
Didayolo merged 4 commits into
developfrom
average-rank

Conversation

@Didayolo

@Didayolo Didayolo commented Apr 23, 2026

Copy link
Copy Markdown
Member

Description

Add a computation column "average rank". It computes the fractional ranking of each sub-column, then the average. It needed to be injected because it involves comparison with other submissions. The score get updated everytime a submission is added or removed from the leaderboard.

Capture d’écran 2026-04-23 à 11 58 38

Issues this PR resolves

A checklist for hand testing

  • From competition.yaml
  • From the editor
  • Add and remove submissions from the leaderboard and check that the average rank is right

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • CircleCi tests are passing
  • Ready to merge

@ihsaan-ullah

Copy link
Copy Markdown
Collaborator

Test 1:

3 submissions with the following scores and rank

Submission Accuracy F1 Score Average Rank
Sub 1 0.90 0.80 1.5
Sub 2 0.70 0.95 2.0
Sub 3 0.80 0.70 2.5
1

When submission 2 removed, the rank changes to 1 and 2

Submission Accuracy F1 Score Average Rank
Sub 1 0.90 0.80 1
Sub 3 0.80 0.70 2
2

@ihsaan-ullah

Copy link
Copy Markdown
Collaborator

Test 2:

3 submissions with the following scores and rank

Submission Accuracy F1 Score Average Rank
Sub 1 0.90 0.85 1.75
Sub 2 0.75 0.90 2.0
Sub 3 0.90 0.70 2.25
Screenshot 2026-06-18 at 3 47 50 PM

When submission 1 removed, the rank changes to 1.5 and 1.5

Submission Accuracy F1 Score Average Rank
Sub 2 0.75 0.90 1.5
Sub 3 0.90 0.70 1.5
Screenshot 2026-06-18 at 3 48 45 PM

@ihsaan-ullah

ihsaan-ullah commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Test 3:

  • Works from the editor
  • Works from competition yaml
  • Adding, removing submissions updates the rank

@ihsaan-ullah

Copy link
Copy Markdown
Collaborator

Test Resources

I have used this competiiton bundle for testing: Avg_Rank_Bundle.zip

The submission for this bundle has the scores which are copied to the leaderboard. Here is a sample submission: Sample_Submissino.zip

@Didayolo

Copy link
Copy Markdown
Member Author

@ihsaan-ullah Thank you for the review!

@Didayolo Didayolo merged commit c85093d into develop Jun 18, 2026
1 check passed
@Didayolo Didayolo deleted the average-rank branch June 18, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Ranking program: organizer provides the code for computation between leaderboard columns

3 participants