Skip to content

Support when tokens have null ranges#1072

Merged
TwitchBronBron merged 8 commits intomasterfrom
null-token-range-support
Mar 7, 2024
Merged

Support when tokens have null ranges#1072
TwitchBronBron merged 8 commits intomasterfrom
null-token-range-support

Conversation

@TwitchBronBron
Copy link
Member

@TwitchBronBron TwitchBronBron commented Feb 13, 2024

Many plugins use Parser.parse() to generate AST used to replace existing code. This causes the generated sourcemaps to be very incorrect. To mitigate this, we should support generating AST without any range information at all. But, lots of our codebase explodes when that happens.

This PR adds a test that verifies most expressions and statements can transpile to proper code without needing tokens. There may be some edge cases, but we've tried to hit as many as possible.

Here's how you can leverage this new functionality:

const tokens = Parser.parse('print hello', { trackLocations: false });

The tokens array will have all the same tokens, but none of them will have location information, making them much safer to inject into AST without messing with sourcemaps.

@TwitchBronBron TwitchBronBron marked this pull request as ready for review March 7, 2024 17:10
@TwitchBronBron TwitchBronBron merged commit f442095 into master Mar 7, 2024
@TwitchBronBron TwitchBronBron deleted the null-token-range-support branch March 7, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant