【SpecInfer】修复 InferenceWithReference 接收率不高的 bug

[Wanglongzhi2001]

Before submitting

• Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

• Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

Bug fixes

PR changes

Others

Description

修复 InferenceWithReference 接收率不高的 bug

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 51.54%. Comparing base (235c24e) to head (78f2833).
Report is 338 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #9880      +/-   ##
===========================================
- Coverage    51.66%   51.54%   -0.12%     
===========================================
  Files          739      741       +2     
  Lines       117426   117904     +478     
===========================================
+ Hits         60668    60775     +107     
- Misses       56758    57129     +371     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.