Skip to content

Feature/storage/monitoring tool#1191

Closed
OhMaley wants to merge 12 commits into
codalab:developfrom
OhMaley:feature/storage/monitoring-tool
Closed

Feature/storage/monitoring tool#1191
OhMaley wants to merge 12 commits into
codalab:developfrom
OhMaley:feature/storage/monitoring-tool

Conversation

@OhMaley

@OhMaley OhMaley commented Oct 18, 2023

Copy link
Copy Markdown
Collaborator

Description

This PR aims to offer a monitoring tool similar to the one available in Codalab.
The following features have been developed:

  • The monitoring tool is only accessible by admins
  • It is accessible under the user menu -> analytics -> storage tab
  • 3 metrics are available
    • Usage history
    • Competitions usage history and details
    • Users usage history and details
  • Metrics can be download using the download buttons
  • The metrics only displays what has been computed by the server beforehand
  • 2 background tasks has been coded:
    • a reset task that get executed every 3 month (based on the value that is set in the settings/base.py). It reset the value of all computed file sizes so they will be re-computed again without any shifting on the next run of the storage analytics task
    • a storage analytic snapshot task that get executed every week (based on the value that is set in the settings/base.py). It measures all files with unset size. Then it measures the storage usage per category (competition, user or admin) and per day. It then check for database <-> storage inconsistency (and created a log file in the logs/ folder). And save everything into the database so it becomes query-able by the front-end.

Issues this PR resolves

#713

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • CircleCi tests are passing
  • Ready to merge

@OhMaley OhMaley added the Enhancement Feature suggestions and improvements label Oct 18, 2023
@Didayolo

Copy link
Copy Markdown
Member

Should we merge this to a new branch on codabench repository to trigger the automatic tests?

@Didayolo Didayolo mentioned this pull request Nov 28, 2023
7 tasks
@Didayolo Didayolo closed this Nov 28, 2023
@Didayolo Didayolo mentioned this pull request Nov 28, 2023
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement Feature suggestions and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants