- Modified symbol_ids HashMap to store tuples of (String, u16) instead of just String
- Updated symbol ID generation to assign numeric IDs sequentially (0 for end symbol, then 1, 2, 3...)
- Changed all symbol_ids access patterns throughout codebase to use tuple destructuring (.0 for string, .1 for numeric)
- Updated node_types.json to use numeric u16 symbol_id instead of String
- Added symbol_id as optional field in NodeTypeJSON struct for tracking grammar symbols
- Threaded symbol_ids HashMap through generate_node_types_json function to populate symbol IDs
- Updated all test assertions to include symbol_id: None for backward compatibility
- Consolidated grammar processing logic into new `introspect_grammar` function
- Removed intermediate `GeneratedParser` and `JSONOutput` structs in favor of direct `GrammarIntrospection` struct
- Simplified code generation flow by separating grammar analysis from code rendering
- Moved symbol ID generation logic out of renderer initialization into standalone function
- Extracted sanitize_identifier and metadata_for_symbol as reusable helper functions
- Symbol IDs now computed before rendering and passed to renderer constructor
Problem: embedding the CLI version used to generate a parser triggers CI
failures on all grammars for every (patch) release of tree-sitter, even
if there are no actual parser changes.
Solution: do not embed the version; instead rely on whether the update
introduces actual (presumably desirable) changes in the parser to
indicate regeneration is necessary.
It was only used to share two constants, and balloons its dependencies.
This also makes `generate_parser_for_grammar` work in wasm.
(Tested in `wasm32-wasip2` in wasmtime with the json grammar,
`wasm32-unknown-unknown` running in the same setup exited successfully
so I'm pretty confident it works as well)
Co-authored-by: Amaan Qureshi <contact@amaanq.com>
This adds an `--evaluate-only` option to `tree-sitter generate`
so that it only does the evaluation of `grammar.js` to
`src/grammar.json`, without continuing on with the generation of
`src/parser.c` and related files.
It's a follow-up to #4580.
