Problem using mldesigner Output

[fredrike]

• mldesigner:
• 0.1.0b8:
• Linux f170461 5.4.0-1074-azure 2013-03-20 Version 0.6.2 Updates #77~18.04.1-Ubuntu SMP Wed Mar 30 15:36:02 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux:
• **Python 3.8.13 (default, Mar 28 2022, 11:38:47) **:

Describe the bug
According to mldesigner/_input_output.py there should be support for path and mode parameters for Output like the example below:

from mldesigner import Output, command_component

@command_component(
    name="test",
    version="1",
    environment=dict(
        conda_file="./dependencies/conda.yaml",
        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220915.v1",
    ),
)
 def  test(
    outpus: Output(
        type="uri_folder",
        path="azureml://datastores/name/paths/path/",
        mode="rw_mount",
    ),
 ):
    pass

This does however yeild the following error:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/mldesigner/_component.py:209, in command_component.<locals>.component_func_decorator(func)
    208 try:
--> 209     wrapper.component = executor.component
    210 except Exception as e:  # pylint: disable=broad-except

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/mldesigner/_dependent_component_executor.py:99, in DependentComponentExecutor.component(self)
     98 except Exception as e:
---> 99     raise e

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/mldesigner/_dependent_component_executor.py:82, in DependentComponentExecutor.component(self)
     80 from azure.ai.ml.entities._component.component_factory import component_factory
---> 82 component = component_factory.load_from_dict(
     83     _type=NodeType.COMMAND,
     84     data={**entity_args, **io_properties},
     85     context={BASE_PATH_CONTEXT_KEY: Path("./")},
     86     _source=ComponentSource.MLDESIGNER,
     87 )
     88 return component

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/azure/ai/ml/entities/_component/component_factory.py:145, in _ComponentFactory.load_from_dict(self, data, context, _type, **kwargs)
    141 new_instance = create_instance_func()
    142 new_instance.__init__(
    143     yaml_str=kwargs.pop("yaml_str", None),
    144     _source=kwargs.pop("_source", ComponentSource.YAML_COMPONENT),
--> 145     **(create_schema_func(context).load(data, unknown=INCLUDE, **kwargs)),
    146 )
    147 return new_instance

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/marshmallow/schema.py:722, in Schema.load(self, data, many, partial, unknown)
    702 """Deserialize a data structure to an object defined by this Schema's fields.
    703 
    704 :param data: The data to deserialize.
   (...)
    720     if invalid data are passed.
    721 """
--> 722 return self._do_load(
    723     data, many=many, partial=partial, unknown=unknown, postprocess=True
    724 )

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/marshmallow/schema.py:909, in Schema._do_load(self, data, many, partial, unknown, postprocess)
    908     self.handle_error(exc, data, many=many, partial=partial)
--> 909     raise exc
    911 return result

ValidationError: {'outputs': defaultdict(<class 'dict'>, {'yearly_volumes': {'value': [{'path': ['Unknown field.']}, {'type': ["Value 'uri_folder' passed is not in set ['boolean', 'integer', 'number', 'string']"], 'path': ['Unknown field.']}]}})}

The above exception was the direct cause of the following exception:

ComponentDefiningError                    Traceback (most recent call last)
Input In [1], in <cell line: 12>()
      9 from azure.ai.ml.constants import AssetTypes
     11 # load component function from component python file
---> 12 from create_yearly_volumes import test
     13 from write_profiled_volumes import write_profiled_volumes
     14 create_profiled_volumes_component = load_component(source="./create_profiled_volumes.yaml")

File /mnt/batch/tasks/shared/LS_root/mounts/clusters/f170461/code/Users/F17046/PFS/EPS/eps_co_forecast_jobs/profiled_forecast_volume/create_yearly_volumes.py:90, in <module>
     79 @command_component(
     80     name="test",
     84     environment=dict(
     85         conda_file="./dependencies/conda.yaml",
     86         image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220915.v1",
     87     ),
     89 )
---> 90 def test(
     91     yearly_volumes: Output(
     92         type="uri_folder",
     93         path="azureml://datastores/name/paths/path/",
     94         mode="rw_mount",
     95     ),
     96 ):
     97     pass

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/mldesigner/_component.py:211, in command_component.<locals>.component_func_decorator(func)
    209         wrapper.component = executor.component
    210     except Exception as e:  # pylint: disable=broad-except
--> 211         raise ComponentDefiningError(name=entity_args["name"], cause=str(e)) from e
    212 return wrapper

ComponentDefiningError: Defining component 'test' failed: {'outputs': defaultdict(<class 'dict'>, {'yearly_volumes': {'value': [{'path': ['Unknown field.']}, {'type': ["Value 'uri_folder' passed is not in set ['boolean', 'integer', 'number', 'string']"], 'path': ['Unknown field.']}]}})}.

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Compute:0.08512443,Event Hubs:0.08366934,Azure.Core:0.051539335'

[xiangyan99]

Thanks for the feedback, we’ll investigate asap.

[xiangyan99]

@azureml-github

[brynn-code]

@fredrike The root cause is we are using one Output class for both defining a component and consuming a component, 'path' is a field which can only be used when consuming the component, so at the end, error message said: 'path': ['Unknown field.'].

Hi @fredrike, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

[fredrike]

/unresolve