[QNN EP] Passthrough EP Parameters in Node by joncamp · Pull Request #23468 · microsoft/onnxruntime

joncamp · 2025-01-23T00:47:51Z

Description

The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the webgpu provider and parse/populate the options accordingly.

Existing defaults are preserved, such that if options are not set the prior behavior will persist.

Motivation and Context

During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development.

Update forked repository to latest

HectorSVC · 2025-01-24T21:45:04Z

/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline

HectorSVC · 2025-01-24T21:45:17Z

/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed

HectorSVC · 2025-01-24T21:45:34Z

/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline

azure-pipelines · 2025-01-24T21:45:45Z

Azure Pipelines successfully started running 5 pipeline(s).

azure-pipelines · 2025-01-24T21:45:48Z

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines · 2025-01-24T21:45:55Z

Azure Pipelines successfully started running 5 pipeline(s).

fs-eire · 2025-01-24T22:07:25Z

Sorry for the late review.

I think we should better first define the JavaScript API for QNN here, then in this PR parse/read them accordingly.

fs-eire · 2025-01-24T22:21:40Z

Sorry for the late review.

I think we should better first define the JavaScript API for QNN here, then in this PR parse/read them accordingly.

I created #23486. Please help take a look

jywu-mysoft · 2025-01-28T17:00:07Z

Sorry for the late review.
I think we should better first define the JavaScript API for QNN here, then in this PR parse/read them accordingly.

I created #23486. Please help take a look

thanks @fs-eire! Can #23486 be merged?

…ault elsewhere. No need to set it here.

fs-eire · 2025-01-29T00:26:28Z

Sorry for the late review.
I think we should better first define the JavaScript API for QNN here, then in this PR parse/read them accordingly.

I created #23486. Please help take a look

thanks @fs-eire! Can #23486 be merged?

Please let me know if you are good with the change (naming and typing). Please approve the PR if you are OK with it and I will go merge it.

### Description As a pre-requisite of #23468

jywu-mysoft · 2025-02-05T00:35:54Z

/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed

jywu-mysoft · 2025-02-05T00:36:06Z

/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline

jywu-mysoft · 2025-02-05T00:36:17Z

/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline

azure-pipelines · 2025-02-05T00:36:18Z

Azure Pipelines successfully started running 5 pipeline(s).

azure-pipelines · 2025-02-05T00:36:26Z

Azure Pipelines successfully started running 5 pipeline(s).

azure-pipelines · 2025-02-05T00:36:52Z

Azure Pipelines successfully started running 9 pipeline(s).

### Description As a pre-requisite of microsoft#23468

fs-eire · 2025-02-09T05:25:41Z

I checked the implementation and I found that the existing implementation has a few errors.

Since the errors are not only for QNN ( there are also places to be fixed for CoreML and WebGPU), so I made this change. Please refer to 4ce51f3 as a patch to this PR.

### Description As a pre-requisite of #23468

jywu-mysoft · 2025-02-18T21:14:19Z

I checked the implementation and I found that the existing implementation has a few errors.

Since the errors are not only for QNN ( there are also places to be fixed for CoreML and WebGPU), so I made this change. Please refer to 4ce51f3 as a patch to this PR.

@joncamp would you be able to apply @fs-eire 's patch to this PR? let's try to merge it into main asap (for upcoming ORT 1.21 release, code freeze end of week)

fix session option parse

jywu-mysoft · 2025-02-19T20:07:41Z

/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed

jywu-mysoft · 2025-02-19T20:07:57Z

/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline

azure-pipelines · 2025-02-19T20:08:07Z

Azure Pipelines successfully started running 5 pipeline(s).

jywu-mysoft · 2025-02-19T20:08:14Z

/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline

azure-pipelines · 2025-02-19T20:08:37Z

Azure Pipelines successfully started running 5 pipeline(s).

azure-pipelines · 2025-02-19T20:08:37Z

Azure Pipelines successfully started running 9 pipeline(s).

fs-eire

LTGM.

Please test if the code works before merge.

jywu-mysoft · 2025-02-20T01:49:32Z

/azp run Linux GPU CI Pipeline

azure-pipelines · 2025-02-20T01:49:44Z

Azure Pipelines successfully started running 1 pipeline(s).

### Description As a pre-requisite of #23468

### Description The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the `webgpu` provider and parse/populate the options accordingly. Existing defaults are preserved, such that if options are not set the prior behavior will persist. ### Motivation and Context During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development. --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>

### Description As a pre-requisite of #23468

### Description The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the `webgpu` provider and parse/populate the options accordingly. Existing defaults are preserved, such that if options are not set the prior behavior will persist. ### Motivation and Context During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development. --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>

Jon Campbell added 14 commits January 8, 2025 12:51

Merge pull request #1 from microsoft/main

d578127

Update forked repository to latest

Fix QNN options parsing, respecting existing defaults

ed5ea28

Merge branch 'microsoft:main' into main

5fbf54a

Merge branch 'main' into joncamp/qnn_fixes

ecef8b1

Merge branch 'microsoft:main' into main

64450ac

Merge branch 'main' into joncamp/qnn_fixes

e7f13fe

Merge branch 'microsoft:main' into main

5130799

Merge branch 'main' into joncamp/qnn_fixes

fa014ca

Add logging for QNN backend

dc4e0c4

Merge branch 'microsoft:main' into main

2111787

Merge branch 'main' into joncamp/qnn_fixes

f40d146

Merge branch 'microsoft:main' into joncamp/qnn_fixes

2716242

Merge branch 'microsoft:main' into joncamp/qnn_fixes

e840444

Remove extra logging

be55bad

jywu-mysoft requested a review from fs-eire January 23, 2025 01:35

fs-eire mentioned this pull request Jan 24, 2025

[js] update JavaScript API to support QNN EP options #23486

Merged

HectorSVC reviewed Jan 24, 2025

View reviewed changes

Comment thread js/node/src/session_options_helper.cc Outdated

HectorSVC added the ep:QNN issues related to QNN exeution provider label Jan 24, 2025

Per PR feedback, enable_htp_fp16_precision is already set to 1 by def…

429a02f

…ault elsewhere. No need to set it here.

fs-eire added a commit that referenced this pull request Feb 4, 2025

[js] update JavaScript API to support QNN EP options (#23486)

faee912

### Description As a pre-requisite of #23468

jywu-mysoft reviewed Feb 5, 2025

View reviewed changes

Comment thread js/node/src/session_options_helper.cc Outdated

sfatimar pushed a commit to intel/onnxruntime that referenced this pull request Feb 5, 2025

[js] update JavaScript API to support QNN EP options (microsoft#23486)

aa410ba

### Description As a pre-requisite of microsoft#23468

sfatimar pushed a commit to intel/onnxruntime that referenced this pull request Feb 5, 2025

[js] update JavaScript API to support QNN EP options (microsoft#23486)

4d760e3

### Description As a pre-requisite of microsoft#23468

fix session option parse

4ce51f3

ashrit-ms pushed a commit that referenced this pull request Feb 11, 2025

[js] update JavaScript API to support QNN EP options (#23486)

d9e6a60

### Description As a pre-requisite of #23468

Jon Campbell added 2 commits February 19, 2025 11:45

Merge branch 'microsoft:main' into joncamp/qnn_fixes

8a00e64

Merge pull request #2 from microsoft/fs-eire/qnn_fixes

9aef86f

fix session option parse

fs-eire approved these changes Feb 19, 2025

View reviewed changes

jywu-mysoft merged commit 75cf166 into microsoft:main Feb 20, 2025

joncamp deleted the joncamp/qnn_fixes branch February 24, 2025 16:52

guschmue pushed a commit that referenced this pull request Mar 6, 2025

[js] update JavaScript API to support QNN EP options (#23486)

d67b946

### Description As a pre-requisite of #23468

ashrit-ms pushed a commit that referenced this pull request Mar 17, 2025

[js] update JavaScript API to support QNN EP options (#23486)

e506f9a

### Description As a pre-requisite of #23468

Uh oh!

Conversation

joncamp commented Jan 23, 2025

Description

Motivation and Context

Uh oh!

HectorSVC commented Jan 24, 2025

Uh oh!

HectorSVC commented Jan 24, 2025

Uh oh!

HectorSVC commented Jan 24, 2025

Uh oh!

azure-pipelines Bot commented Jan 24, 2025

Uh oh!

azure-pipelines Bot commented Jan 24, 2025

Uh oh!

azure-pipelines Bot commented Jan 24, 2025

Uh oh!

fs-eire commented Jan 24, 2025

Uh oh!

fs-eire commented Jan 24, 2025

Uh oh!

Uh oh!

jywu-mysoft commented Jan 28, 2025

Uh oh!

fs-eire commented Jan 29, 2025

Uh oh!

jywu-mysoft commented Feb 5, 2025

Uh oh!

jywu-mysoft commented Feb 5, 2025

Uh oh!

jywu-mysoft commented Feb 5, 2025

Uh oh!

azure-pipelines Bot commented Feb 5, 2025

Uh oh!

azure-pipelines Bot commented Feb 5, 2025

Uh oh!

azure-pipelines Bot commented Feb 5, 2025

Uh oh!

Uh oh!

fs-eire commented Feb 9, 2025

Uh oh!

jywu-mysoft commented Feb 18, 2025

Uh oh!

jywu-mysoft commented Feb 19, 2025

Uh oh!

jywu-mysoft commented Feb 19, 2025

Uh oh!

azure-pipelines Bot commented Feb 19, 2025

Uh oh!

jywu-mysoft commented Feb 19, 2025

Uh oh!

azure-pipelines Bot commented Feb 19, 2025

Uh oh!

azure-pipelines Bot commented Feb 19, 2025

Uh oh!

fs-eire left a comment

Choose a reason for hiding this comment

Uh oh!

jywu-mysoft commented Feb 20, 2025

Uh oh!

azure-pipelines Bot commented Feb 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants