When executing a Playground in braintrust, remote eval task is always passed an empty metadata map for each dataset, even if those datasets have metadata.
Reproduction steps:
- Create a dataset with metadata
- Execute dataset from Playground, observe no metadata available in remote eval task
- Select data set in Playground, run it individually, observe metadata IS available in remote eval task
Expected:
Each dataset passed to a remote eval task processor function should have its metadata available to that processor regardless of how the execution was triggered
When executing a Playground in braintrust, remote eval task is always passed an empty metadata map for each dataset, even if those datasets have metadata.
Reproduction steps:
Expected:
Each dataset passed to a remote eval task processor function should have its metadata available to that processor regardless of how the execution was triggered