Skip to content

Commit 2776807

Browse files
Docs: Add docs explaining baseline submission process (Cloud-CV#4764)
Co-authored-by: Rishabh Jain <rishabhjain2018@gmail.com>
1 parent 356b526 commit 2776807

1 file changed

Lines changed: 66 additions & 1 deletion

File tree

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,66 @@
1-
# Baseline Submissions
1+
# Baseline Submissions
2+
3+
A **Baseline Submission** is a special submission created by a Challenge Host that acts as a reference point on the leaderboard.
4+
Until a submission is marked as a baseline, it does not appear on the leaderboard for participants to compare their submissions.
5+
6+
---
7+
8+
## Why Set a Baseline?
9+
10+
- Helps participants understand the minimum standard they should surpass.
11+
- Provides an official benchmark for participants to improve upon.
12+
- Makes the leaderboard informative and engaging by showing a reference performance.
13+
14+
---
15+
16+
## How to Make a Submission Baseline
17+
18+
Follow these steps to create, upload and mark a submission Baseline in EvalAI:
19+
20+
### Step 1: Prepare Your Baseline Submission
21+
22+
- Develop a solution or model that represents a reasonable starting point or minimal benchmark for the challenge.
23+
- Ensure your submission complies with the challenge’s submission format and requirements (e.g., correct output files, file format, metadata).
24+
25+
### Step 2: Submit the Baseline
26+
27+
- Log in to EvalAI as the Challenge Host.
28+
- Navigate to the **Challenge Dashboard** and select your challenge.
29+
- Go to the **Submit** tab.
30+
- Upload your baseline submission just like a regular participant submission.
31+
32+
### Step 3: Open the My Submissions Tab
33+
34+
- After the submission is uploaded, go to the **My Submissions** tab of your challenge.
35+
36+
### Step 4: Locate the Submission
37+
38+
- Find the submission you just uploaded and want to mark as baseline.
39+
40+
### Step 5: Mark the Submission as Baseline
41+
42+
- Once the submission has passed the evaluation successfully, click the checkbox in the **Baseline** column next to the submission entry, .
43+
44+
### Step 6: Confirm Baseline Display
45+
46+
- The baseline submission will now appear on the leaderboard with a "B" badge indicating a Basline submission.
47+
- Participants will see this baseline as a reference when viewing the leaderboard.
48+
49+
---
50+
51+
## Managing Baseline Submissions
52+
53+
- You can have multiple baseline submissions if your challenge requires benchmarks for different metrics or tasks.
54+
- Only Challenge Hosts have permission to mark or unmark submissions as baselines.
55+
- If needed, you can unmark or replace a baseline submission at any time by following the same steps and unchecking previous submission's **Baseline** checkbox and marking a new submission as baseline to update it.
56+
57+
---
58+
59+
### Note:
60+
61+
- Choose a baseline that is meaningful and encourages participants to improve.
62+
- Update baseline submissions periodically if better baseline models become available.
63+
- Communicate clearly to participants what the baseline represents in the challenge overview or description.
64+
65+
66+
---

0 commit comments

Comments
 (0)