[evaluation] refactor: Make RunSubmitterClient default batch client (remove promptflow) - #42243

Merged

kdestin merged 26 commits into

Azure:mainfrom

kdestin:evaluation/remove-promptflow

Jul 30, 2025

Commits on Jul 28, 2025

chore: Remove promptflow dependencies
kdestin
committed
2790915
fix: Filter inputs before calling function in Batch Engine
kdestin
committed
a3dfa6e
refactor: Use enumerate instead of manually keeping track of line number
kdestin
committed
85eecc5
fix,refactor: Unconditionally inject default column mapping from data -> params
kdestin
committed
cafe8ba
refactor: Don't shadow `value` variable in apply_column_mapping_to_lines
kdestin
committed
1e3d7f5
feat: Add support for running aggregations in RunSubmitterClient
kdestin
committed
1d1882a
tests,fix: Don't log duration as a metric
kdestin
committed
53f6a3e
refactor: Rewrite RunSubmitterClient.get_details
kdestin
committed
40f7452
fix: Correct the typing of is_onedp_project
kdestin
committed
00d8b2f
fix,tests: Don't log tokens as metrics
kdestin
committed
0322d0b
fix: Set error message without depending on run storage
kdestin
committed
d58be1c
tests,fix: Fix test_evaluate_invalid_column_mapping
kdestin
committed
5bb01a7
fix,tests: Force PFClient specific tests to use PFClient
kdestin
committed
150d63b
fix,tests: Force CodeClient specific tests to use CodeClient
kdestin
committed
089e568
fix: Improve the ergonomics for picking which client is used
kdestin
committed
d1f0937
feat: Show exception message in run logs
kdestin
committed
06e05c4
fix: For safety evaluation to use codeclient as originally intended
kdestin
committed
07296fc
fix: Don't wrap EvaluationExcpeiton in BatchEngineError
kdestin
committed
6c2d8c0
refactor: Refactor BatchConfig
kdestin
committed
92a4644
fix: Make raising on error configurable for runsubmitterclient
kdestin
committed
3af4f9d
chore: Update changelog
kdestin
committed
b9cad33
fix: Uncomment log_path
kdestin
committed
0e2a560
chore: Add promptflow to dev-requirements.txt
kdestin
committed
5d4968e
fix: Initialize error_message
kdestin
committed
b79f3a0
fix: Get exception instead of batchrunresult
kdestin
committed
9a7c190

Commits on Jul 29, 2025

chore,docs: Clarify changelog
kdestin
committed
be8285d

Navigation Menu

Uh oh!

[evaluation] refactor: Make RunSubmitterClient default batch client (remove promptflow) - #42243

[evaluation] refactor: Make RunSubmitterClient default batch client (remove promptflow)#42243
kdestin merged 26 commits into
Azure:mainfrom
kdestin:evaluation/remove-promptflow

Commits on Jul 28, 2025

chore: Remove promptflow dependencies

fix: Filter inputs before calling function in Batch Engine

refactor: Use enumerate instead of manually keeping track of line number

fix,refactor: Unconditionally inject default column mapping from data -> params

refactor: Don't shadow `value` variable in apply_column_mapping_to_lines

feat: Add support for running aggregations in RunSubmitterClient

tests,fix: Don't log duration as a metric

refactor: Rewrite RunSubmitterClient.get_details

fix: Correct the typing of is_onedp_project

fix,tests: Don't log tokens as metrics

fix: Set error message without depending on run storage

tests,fix: Fix test_evaluate_invalid_column_mapping

fix,tests: Force PFClient specific tests to use PFClient

fix,tests: Force CodeClient specific tests to use CodeClient

fix: Improve the ergonomics for picking which client is used

feat: Show exception message in run logs

fix: For safety evaluation to use codeclient as originally intended

fix: Don't wrap EvaluationExcpeiton in BatchEngineError

refactor: Refactor BatchConfig

fix: Make raising on error configurable for runsubmitterclient

chore: Update changelog

fix: Uncomment log_path

chore: Add promptflow to dev-requirements.txt

fix: Initialize error_message

fix: Get exception instead of batchrunresult

Commits on Jul 29, 2025

chore,docs: Clarify changelog