Hoist special-token lookups out of per-token loops in six slow tokenizers

[ishan-1010]

What this fixes

Fixes #47424. Third instance of the pattern from #46323 and #46535: all_special_tokens / all_special_ids are rebuilt on every property access, and six slow tokenizers read them once per token inside decode or mask loops.

The fix

Hoist the lookup into a local set(...) before each loop. Files: esm (get_special_tokens_mask), marian, m2m_100, speech_to_text, gpt_sw3, siglip (convert_tokens_to_string). 6 files, +12/-6, membership results unchanged (set vs list membership is identical for these strings/ints).

Left out on purpose: wav2vec2 (same pattern, but open PR #46578 edits that line; can follow up after it lands), wav2vec2_phoneme (that check compares a str against int ids and is dead as written; noted in the issue), bartpho (same read but in save_vocabulary, not a hot path).

Numbers

ESM, get_special_tokens_mask on 20k ids (M1, CPU): 26.8 ms before, 0.3 ms after, identical mask.

Tests

python -m pytest tests/models/{esm,marian,m2m_100,speech_to_text,gpt_sw3,siglip}/test_tokenization_*.py
# 167 passed, 117 subtests passed; remaining failures/errors are identical on unmodified main (network/fixture dependent)

Notes

Not a duplicate: the only open PR mentioning all_special_ids (#46574) is an unrelated CodeLlama decode fix; #46578 covers wav2vec2 only. AI-assisted; I reviewed every line and ran the tests and benchmark above.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: esm, gpt_sw3, m2m_100, marian, siglip, speech_to_text

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 29723351253
Result: success | Grafana metrics are not available yet.

[Rocketknight1]

Yeah, this is a clean optimization that I'm happy to merge!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.