This solves a bug on big-endian architectures where the value would be later passed by reference as an elements "view" before being inserted. The issue is it is casted as a void pointer, and when writing uint16_t's of size 1, only 2 of the 4 bytes are written. This is okay for little-endian systems, but not big-endian
This change exposes a new `primary_state_ids` field on the `TSLanguage`
struct, and populates it by tracking the first encountered state with a
given `core_id`. (For posterity: the initial change just exposed
`core_id` and deduplicated within `ts_analyze_query`).
With this `primary_state_ids` field in place, the
`ts_query__analyze_patterns` function only needs to populate its
subgraphs with starting states that are _primary_, since non-primary
states behave identically to primary ones. This leads to large savings
across the board, since most states are not primary.
- Use a proper enum type for quantifiers.
- Drop quantifiers from `TSQueryStep`, which was not used.
- Keep track of the captures introduced during a pattern parse, and
apply the quantifier for the pattern to the captures that were
introduced by the pattern or any sub patterns.
- Use 'quantifier' instead of 'suffix'.
Profiling the `ts_query__analyze_patterns` function shows that it
spends a lot of time copying items in its various state sets. These
state sets are kept sorted, and the items are fairly large, so any time
that we insert new entries near the front of the array, a lot of calls
to memcpy must occur.
In advance of more sophisticated rework, one easy win is to hide the
large `AnalysisStateSet` objects behind pointers, so that the size of
each item in the list goes from 68 to 8 bytes, and add an object pool to
reuse allocations. This shows a significant performance improvement for
grammars that have a lot of states in them.
* Fix bugs related to named wildcard patterns vs regular wildcard patterns.
* Fix handling of extra nodes during query analysis. Previously, the
expected child_index was updated incorrectly after an extra node,
leading to false "impossible pattern" errors.
* Refine logic for avoiding unnecessary state-splitting due to fallible steps.
Compute *two* different analysis results related to step fallibility:
* `root_pattern_guaranteed` which, like before, summarizes whether the
entire pattern is guaranteed to match once this step is reached.
* `parent_pattern_guaranteed` - which just indicates whether the
immediate parent pattern is guaranteed. This is now used when
deciding whether it's necessary to split a match state.
When descending into a hidden child rule, the current stack entry
can be reused if it is currently at the end of its rule.
This fixes a test failure when analyzing a Ruby query. The
failure was introduced due to some changes to the Ruby grammar.
This optimization allows us to impose a _smaller_ limit on
the stack size, which should make query analysis faster and
more memory-efficient.
