Skip to content

Feature/storage/monitoring tool#1247

Merged
Didayolo merged 12 commits into
codalab:monitoring-toolfrom
OhMaley:feature/storage/monitoring-tool
Nov 28, 2023
Merged

Feature/storage/monitoring tool#1247
Didayolo merged 12 commits into
codalab:monitoring-toolfrom
OhMaley:feature/storage/monitoring-tool

Conversation

@Didayolo

Copy link
Copy Markdown
Member

Merge the PR into a branch in our own codalab/codabench repository.

Original PR:


Description

This PR aims to offer a monitoring tool similar to the one available in Codalab.
The following features have been developed:

  • The monitoring tool is only accessible by admins
  • It is accessible under the user menu -> analytics -> storage tab
  • 3 metrics are available
    • Usage history
    • Competitions usage history and details
    • Users usage history and details
  • Metrics can be download using the download buttons
  • The metrics only displays what has been computed by the server beforehand
  • 2 background tasks has been coded:
    • a reset task that get executed every 3 month (based on the value that is set in the settings/base.py). It reset the value of all computed file sizes so they will be re-computed again without any shifting on the next run of the storage analytics task
    • a storage analytic snapshot task that get executed every week (based on the value that is set in the settings/base.py). It measures all files with unset size. Then it measures the storage usage per category (competition, user or admin) and per day. It then check for database <-> storage inconsistency (and created a log file in the logs/ folder). And save everything into the database so it becomes query-able by the front-end.

Issues this PR resolves

#713

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • CircleCi tests are passing
  • Ready to merge

@Didayolo Didayolo merged commit 896a3cf into codalab:monitoring-tool Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants