Skip to content

Evaluate(): Cannot import data from fields with numbers in their names #40424

Description

@msa2984
  • Package Name: Azure AI Evaluation (azure-ai-evaluation)
  • Package Version: 1.4.0, 1.5.0
  • Operating System: Windows, WSL, Ubuntu (we have multiple environments)
  • Python Version: 3.12

Describe the bug
Our team is trying to update the version of azure-ai-evaluation we are referencing from 1.1.0 to 1.5.0. When I do this, I get an exception:

azure.ai.evaluation._exceptions.EvaluationException: (UserError) Unexpected references detected in 'column_mapping'. Ensure only ${target.} and ${data.} are used.

I believe it traces to the regex here: 487020b#diff-64960c5664f00223809f9c1584a47a44e466f3aec97427367d55f6e7b1c7211dR527-R528

Testing this regex via pythex,

${data.test_notes} matches
${data.test8_notes} does not match.
To Reproduce
Steps to reproduce the behavior:

  1. Create a data.jsonl file which has at least one field with a number in its name.
  2. When calling evaluate, with an evaluator specify a column mapping which uses that field.
  3. Run the code and you will reproduce the same behavior.

Expected behavior
I can support column mappings with numbers in their name.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

ClientThis issue points to a problem in the data-plane of the library.EvaluationIssues related to the client library for Azure AI EvaluationService AttentionWorkflow: This issue is responsible by Azure service team.customer-reportedIssues that are reported by GitHub users external to the Azure organization.issue-addressedWorkflow: The Azure SDK team believes it to be addressed and ready to close.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions