Skip to content

spec.md

Latest commit

 

History

History
104 lines (93 loc) · 4.36 KB

spec.md

File metadata and controls

104 lines (93 loc) · 4.36 KB

The DeepInstruments spec 22 dec 2015

A. Audio features 1. [DONE] Review silence detection. 2. [DONE] Convert features to float32. 3. [CLOSED] Generate silenced frames in test set. 4. [DONE] Review perceptual loudness reference in get_X 5. [DONE] Explicitly ignore WavFileWarning 6. For standardization, only collect the mean and variance of activated frames

B. Deep learning 1. [DONE] Write Graph model without Z 2. [DONE] Make it a function in module "learning" 3. [DONE] Solve core dump 4. [DONE] Install bleeding-edge Keras 5. [DONE] Train on categorical cross-entropy 6. [DONE] Write data generator 7. Add Z supervision 8. [DONE] Report mean and variance of loss after each epoch

C. Pitch supervision 1. [CLOSED] Get Gt samples for RWC 2. [DONE] Check MIDI offsets in RWC dict 3. [DONE] Write conversion from MIDI to ConvNet axis. 4. [DONE] Patch rankings for The Districts, Vermont 5. [DONE] Extract Z

D. Evaluation 1. [DONE] Write class-based accuracy measure 2. [DONE] Write callbacks to monitor test error 2. [DONE] Integrate the pipeline into a function so that the whole experiment can be ran in one step. 3. [DONE] Measure class imbalance. How many decision windows per class ? 4. Use MIR metrics for multi-label classif. 5. [DONE] Make a 80/20 file-based split for the retained instruments.

E. Display 1. Export filters from conv1 as images for the three experiments. Are they learned note models ? 2. Make a figure for the architecture. 3. Make a figure for the duration of training set and test set for every instrument in single-label dataset.

F. Dataset

  1. [DONE] Get the full MedleyDB dataset
  2. [CLOSED] Update wrangling so that it lists files, not classes
  3. [DONE] Restrict to a certain number of classes
  4. [DONE] Take the max of stems activations that play the same instrument
  5. [DONE] Write a function that outputs Y from the Medley instrument activations, called by generator
  6. [DONE] Upload MedleyDB on di and cerfeuil
  7. [DONE] Extract annotated vs non-annotated files for single-label classes
  8. [DONE] If there are several stems of the same instrument in a given track, discard non-annotated stems from test set
  9. [DONE] Separate singers between training set and test set to avoid artist bias
  10. [DONE] Report misnomer of CroqueMadame_Pilot(Lakelot)_ACTIVATION_CONF.lab
  11. [DONE] Make a patch script in _init.py to handle all misnomers
  12. [DONE] Discard overdrive, shoegaze, bleed and inactivity in clean electric guitar
  13. [DONE] Use version control for the medleydb-single-instruments derived dataset
  14. [DONE] Remove vocal FX tracks
  15. [DONE] Remove first and last chunk (half-silent by definition) of every track

G. Single-label classification

  1. [DONE] Write get_activation
  2. [DONE] Write get_indices (with boundary trimming)
  3. [DONE] Write get_melody
  4. [DONE] Memoize training X with joblib
  5. [DONE] Write a dedicated generator
  6. [DONE] Standardize X in the generator
  7. [DONE] Train deep neural network on X and Y
  8. [DONE] Memoize test X with joblib
  9. Report class-wise accuracy with error bars

H. Descriptors + Random forests baseline

  1. [DONE] Compute MFCCs on the training data
  2. [DONE] Also Delta and Delta-Delta MFCCs
  3. [DONE] Also centroid, bandwidth, contrast, rolloff
  4. [DONE] Generate half-overlapping chunks of X
  5. [DONE] Summarize with mean and variance over chunks
  6. [DONE] Generate Y's as integer classes
  7. [DONE] Same in test set
  8. [DONE] Run scikit-learn's random forest on it
  9. [DONE] Report class-wise accuracy with error bars
  10. [DONE] Discard clean guitar and male singer
  11. [DONE] Bugfix half-overlapping chunks
  12. [DONE] More Tp, Cl, Fl, Pn, and Vl examples (from solosDb)

I. Structured validation

  1. Extract the stem folder of each chunk path
  2. Assign votes to a dict where stems are keys
  3. Get the true class of each stem
  4. Write a systematic structured evaluator

J. Reproducibility

  1. List all operations that are necessary
  2. Review this list on gentiane

K. Scattering transform

  1. [DONE] Write function get_paths in MATLAB
  2. [DONE] Compute joint scattering features
  3. Compute plain scattering features
  4. Compute spiral scattering features
  5. Review the importance of log compression
  6. [DONE] Export in HDF5 from MATLAB to Python
  7. Check that paths are ordered like in Python
  8. Load HDF5, train RF, report accuracy