Skip to content

Runaway whitespace fix#2

Closed
will-exaforce wants to merge 0 commit into
mainfrom
runaway-fix
Closed

Runaway whitespace fix#2
will-exaforce wants to merge 0 commit into
mainfrom
runaway-fix

Conversation

@will-exaforce

@will-exaforce will-exaforce commented Jun 18, 2026

Copy link
Copy Markdown

This PR improves the output generated by the model and the model output normalization when llm is enabled.

This was part of an effort to address issues I observed while validating SkillSpector LLM support locally.

All tests pass locally. I'll get another PR up later to add the proper workflow. In the meantime:

627 passed, 12 skipped, 26 deselected, 1 warning in 1.86s

The benchmark tool was used to compare accuracy and performance against main

Finding summary

Comparison performed across 50 (or all if < 50) samples of each type of data set in https://github.com/lxyeternal/MalSkillBench/tree/main/Dataset.

Performance

runaway-fix completed ~30% faster
runaway-fix - 35:07
main - 46:54

Errors

runaway-fix:

  • None

main:

  • 15 (All timeout)
  • Impacted files
    • 7 of the 50 sampled malicious skills
    • 4 of the 50 sampled malicious python json files

False Negatives

Unchanged across PRs

False Positives

runaway-fix (5):

  • Skills/benign/4chan-reader
  • Skills/benign/abn-skill
  • Skills/benign/aaveclaw
  • Skills/benign/aap-passport
  • Skills/benign/2captcha

main (1):

  • Skills/benign/abn-skill

@will-exaforce
will-exaforce requested review from pupapaik and smoy June 18, 2026 18:04
@will-exaforce
will-exaforce changed the base branch from main to sync-upstream June 18, 2026 19:43
@will-exaforce
will-exaforce changed the base branch from sync-upstream to main June 18, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants