Supporting custom frameworks in SageMaker?? #3
Description
Thank you for shipping this great product!!!
After reading the documents, I understood SageMaker is fully extensible by docker container. And scikit_bring_your_own example clearly shows users how custom python codes can be applied to SageMaker.
However, when I wanted to support custom framework (like torch, chainer, etc.), how to do that effectively?? To achieve this, they possibly need to write notebook side and container side implementations.
On notebook side, I think mxnet support in sagemaker-python-sdk can be really helpful reference.
However, on container side, I couldn't find such reference. But, there would need a lot of common codes for example:
- download user assets(training data, user codes) from s3 to local filesystem
- loading custom user codes to train and save models
- spawning server via wsgi or similar libraries
- detecting cpu/gpu feature
- etc.
I imagined these codes might exist in MXNet, tensorflow support, too. Then, I found SageMakerContainerSupport package in sagemaker-mxnet-py2-cpu:1.0 container. This package defines very useful abstract classes to support custom frameworks for everybody who wants to do it.
Of course, I can write similar codes by myself, however, in OSS nature, I'm really happy if you could open this package to the public. Did you have such plan??