199 show space bytes#263
Conversation
changing html
need cleanup
We now see Collections, Datasets, Files, Bytes per space and Users in right column
max-zilla
left a comment
max-zilla
left a comment
There was a problem hiding this comment.
Left a comment about possibly optimizing the logic of the counting. If it has already been discussed and this is the best way, just let me know.
| files.get(ds_f) match { | ||
| case Some(file) => { | ||
| files.getBytes(file.id) match { | ||
| case Some((stream, name, filetype, bytes)) => { |
There was a problem hiding this comment.
My concern with this is that it could be very slow on large spaces with many files, getting every dataset and every file individually... can we use some of the work done before these calls:
val datasetsInSpace = datasets.listSpace(size, id.toString(), user)
val spaceBytes : Long = getBytesPerSpace(id, user.get)
val spaceFiles : Integer = getFilesPerSpace(id, user.get)
...instead of calling datasets.listSpace again for example?
There was a problem hiding this comment.
I think this is now fixed by caching the spaceBytes as part of the Space object. When datasets are added or files added, the bytes increment, or decrement if removed. So there won't be these calls anymore.
| spaceFiles | ||
| } | ||
|
|
||
| private def getBytesPerSpace(spaceId: UUID, user: models.User) : Long = { |
There was a problem hiding this comment.
this will be slow if there is a few million files in a space, can we cache this at the space level? We already save:
collectionCount: 0,
datasetCount: 5,
userCount: 22,
There was a problem hiding this comment.
I've changed this so that it's another field in the Space object. It gets incremented/decremented as files/datasets are added or deleted.
There was a problem hiding this comment.
I see where that field is implemented and changed which is good - but on the main space page you're still calling this block of code to calculate dynamically right? collections/Spaces line 656 - you should be able to check the cached property of the space instead now I think?
| <div class="panel-body"> | ||
| <ul class="list-group"> | ||
|
|
||
| @* <li class="list-group-item" title="@humanReadableNumber(spacesCount) @Messages("spaces.title")">*@ |
There was a problem hiding this comment.
if not used, please remove commented out code.
There was a problem hiding this comment.
This is now removed.
# Conflicts: # CHANGELOG.md
| spaceFiles | ||
| } | ||
|
|
||
| private def getBytesPerSpace(spaceId: UUID, user: models.User) : Long = { |
There was a problem hiding this comment.
I see where that field is implemented and changed which is good - but on the main space page you're still calling this block of code to calculate dynamically right? collections/Spaces line 656 - you should be able to check the cached property of the space instead now I think?
|
@max-zilla will fix that and make another commit |
|
@max-zilla I fixed the part on app/controllers. That should be done, I do need to test and commit the update script. |
|
Also committed the db update for adding space bytes to existing spaces. |
robkooper
left a comment
robkooper
left a comment
There was a problem hiding this comment.
I will make some changes, this breaks if the file is "imported" and not uploaded. This also breaks in case of large databases (cursor times out).
- no datasets in space would result in not writing out count - very large databsae would have cursor timeout - simplified logic to count bytes
…lowder into 199-show-space-bytes
* make space layout more like datasets/files * update changelog - undo removal - add comment + link to issue for space stats - comment about new layout * Lowered margin from 20px to 15px for button links. Co-authored-by: Luigi Marini <lmarini@illinois.edu>
4 participants
Description
the statistics view for spaces now shows bytes per space.
Review Time Estimate
Types of changes
Checklist: