Skip to content

Enable mklml openmp - #1480

Closed
snnn wants to merge 3 commits into
masterfrom
snnn/enable_mklml_openmp
Closed

Enable mklml openmp#1480
snnn wants to merge 3 commits into
masterfrom
snnn/enable_mklml_openmp

Conversation

@snnn

@snnn snnn commented Jul 24, 2019

Copy link
Copy Markdown
Contributor

Description:
Allow people build onnxruntime with openmp and mklml both.

Motivation and Context
We thought they have conflict, indeed they don't. It was just because we choose the wrong library to link.

  • Why is this change required? What problem does it solve?

On my machine, the mlperf resnet50 model,

(will post a full perf comparison matrix here. So far what I saw was pretty good)

  • If it fixes an open issue, please link to the issue here.

Below is the perf data for OptimizationLevel= 3 and threadcount = 1 , with mklml

model With OpenMP Without OpenMP DIFF
bidaf 0.057664 0.011586 397
bvlc_googlenet 0.0241 0.019951 20
coreml_AgeNet_ImageNet 0.008801 0.006723 30
inception_resnet_v2 0.104942 0.122013 -14
inception_v2 0.015422 0.013983 10
mlperf_mobilenet 0.00294 0.004371 -33
mlperf_resnet 0.021569 0.025979 -17
mlperf_ssd_mobilenet_300 0.02712 0.026646 1
mlperf_ssd_resnet34_1200 1.11819 1.08544 3
resnet50 0.028572 0.023976 19
resnet_v2_101 0.109916 0.110542 -1
resnet_v2_152 0.159811 0.159872 -1
resnet_v2_50 0.061231 0.062894 -3
squeezenet 0.003029 0.002924 3

Changming Sun and others added 2 commits July 23, 2019 18:32
@snnn
snnn requested a review from a team as a code owner July 24, 2019 02:27
Comment thread cmake/CMakeLists.txt
set(ONNX_ML 1)

if(onnxruntime_USE_OPENMP)
if(onnxruntime_USE_OPENMP AND WIN32)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows, it should be possible to only use Intel's OMP library (excluding VCOMP) when OMP and MKLML are both enabled. by using /link /nodefaultlib:vcomp libiomp5md.lib
see https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-using-the-openmp-libraries#C28B029B-7E9F-46D7-8C84-38B0683236B9

@xkszltl

xkszltl commented Jul 24, 2019

Copy link
Copy Markdown
Contributor

Great!
Just FYI if you use mkl_rt from full MKL, you can even do this at runtime with mkl_set_threading_layer : https://software.intel.com/en-us/mkl-linux-developer-guide-dynamically-selecting-the-interface-and-threading-layer

@snnn snnn closed this Jul 24, 2019
@snnn

snnn commented Jul 24, 2019

Copy link
Copy Markdown
Contributor Author

Need to hold on this PR.
If mklml is not better than mlas, we should remove this build option, instead of repairing it.

@yuslepukhin
yuslepukhin deleted the snnn/enable_mklml_openmp branch August 6, 2019 17:03
@WilliamZhaoz

Copy link
Copy Markdown

Is the unit of performance inference time (second)?

@snnn

snnn commented Oct 8, 2019

Copy link
Copy Markdown
Contributor Author

Hi @WilliamZhaoz yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants