Hyper-parameter tuning

[xusun]

Is there a plan to support hyper-parameter tuning as pipeline step?

[rauhs]

See #416

[Zruty0]

Hyper-parameter tuning (or 'sweeping', which is the term we use in the library) is definitely something we plan to support.

For something as trivial as a grid search, I feel like we should enable a code snippet like this (currently this is not working code, but we will likely add this kind of constructor):

foreach (float l2param in new[] {0, 0.1, 0.01, 1e-3, 1e-3}) 
    foreach (float lr in new[] {0.1, 0.01, 1e-3})
    {
        var model = new AveragedPerceptronTrainer(l2param, learningRate: lr)
            .Train(trainingData);
        var metrics = BinaryClassificationMetrics.EvaluateModel(model, validationData);
        if (metrics.AUC > bestMetrics?.AUC ?? 0.5)
        {
            bestMetrics = metrics;
            bestModel = model;
        }
    }

For more complicated sweeping algorithms, we will likely provide a 'sweeper' component, which can be inserted into the 'training pipeline', or used standalone.

[abgoswam]

DRI RESPONSE : Assigning to Backlog project. This is something we plan to support in future

[gilnahmias]

Need an equivalent of https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/tune-model-hyperparameters to ease transition from Azure ML Studio

[codemzs]

We plan to move AutoML to our master branch in the coming weeks, it is currently in a feature branch.