traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
2002 commits 16 branches 98 tags 37 MiB
Commit graph

74 commits

Author SHA1 Message Date
Max Brunsfeld
5372a81947 Simplify treatment of rules that are always aliased one way 2018-08-30 09:57:22 -07:00
Max Brunsfeld
a24f7764d8 Get rid of string_input source file 
There's not much in there now.
 2018-06-19 11:27:55 -07:00
Max Brunsfeld
6fca8f2f4d Make ts_compile_grammar take an optional log file, start logging to it 2018-05-24 16:01:14 -07:00
Max Brunsfeld
9c1e82a7ea ⬆️ utf8proc, use new UTF8PROC_STATIC macro 2018-05-18 10:25:55 -07:00
Max Brunsfeld
e75ecd1bb1 Rework API completely 2018-05-11 10:46:13 -07:00
Max Brunsfeld
35510a612d Rename Tree -> Subtree 2018-05-10 15:11:14 -07:00
Max Brunsfeld
666dfb76d2 Remove document parameter from ts_node_type, ts_node_string 
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
 2018-05-09 16:47:47 -07:00
Max Brunsfeld
186f70649c Consolidate the unify for detecting conflicting tokens 2018-03-28 10:03:09 -07:00
Phil Turnbull
f08402014b Fuzz build should use CFLAGS when linking 2017-09-15 10:06:17 -04:00
Phil Turnbull
c57f8ae0a6 Allow fuzzer CFLAGS to be overridden by envrionment 2017-09-14 11:04:19 -04:00
Max Brunsfeld
d291af9a31 Refactor error comparisons 
* Deal with mergeability outside of error comparison function
* Make `better_version_exists` function pure (don't halt other versions
as a side effect).
* Tweak error comparison logic

Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2017-09-13 16:38:15 -07:00
Max Brunsfeld
9d668c5004 Move incompatible token map into LexTableBuilder 2017-08-31 15:46:37 -07:00
Max Brunsfeld
573b5f3671 Pass LexTableBuilder to ParseTableBuilder 2017-08-25 15:57:50 -07:00
Max Brunsfeld
ced5521014 Fix UTF8PROC_EXPORTS typo 2017-08-07 14:46:11 -07:00
Max Brunsfeld
67fcbba864 Define UTF8PROC_EXPORTS for both compiler and runtime targets 2017-08-07 13:10:41 -07:00
Max Brunsfeld
cb24133c64 Define UTF8PROC_EXPORTS so that utf8proc builds right on windows 2017-08-07 12:44:33 -07:00
Max Brunsfeld
46ec0804f8 Move tree comparison function to a separate translation unit 2017-08-04 11:03:14 -07:00
Phil Turnbull
2155505dcd Enabled Fortify for libruntime.a 
There is currently no code in libruntime.a that Fortify can protect so this
doesn't actually change the generated assembly. However, we may add code in the
future that Fortify can protect.

clang defines `_FORTIFY_SOURCE` by default so we need to undefine then redefine
it to avoid build errors.
 2017-07-17 14:15:08 -07:00
Max Brunsfeld
34279257f9 Merge pull request #91 from tree-sitter/libFuzzer 
Add support for fuzzing with libFuzzer
 2017-07-17 11:43:01 -07:00
Phil Turnbull
798ef5e4dc Add libFuzzer support 
This adds support for fuzzing tree-sitter grammars with libFuzzer. This
currently only works on Linux because of linking issues on macOS. Breifly, the
AddressSanitizer library is dynamically linked into the fuzzer binary and
cannot be found at runtime if built with a compiler that wasn't provided by
Xcode(?). The runtime library is statically linked on Linux so this isn't a
problem.
 2017-07-14 13:50:41 -07:00
Max Brunsfeld
b3a72954ff Introduce RENAME rule type 2017-07-13 17:17:22 -07:00
Max Brunsfeld
76e35fd037 Enable address sanitizer when running tests on mac 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-27 14:54:20 -07:00
Max Brunsfeld
79ffc20be1 Remove RTTI flag in gyp file 2017-03-17 13:31:35 -07:00
Max Brunsfeld
db4b9ebc7c Implement Rule as a union rather than an abstract base class 2017-03-17 13:29:31 -07:00
Max Brunsfeld
64e9230071 Use LexTableBuilder to detect conflicts between tokens more correctly 2017-03-08 12:47:38 -08:00
Max Brunsfeld
abf8a4f2c2 🎨 2017-03-01 22:15:26 -08:00
Max Brunsfeld
686dc0997c Avoid introducing certain lexical conflicts during parse state merging 
The current pretty conservative approach is to avoid merging parse states which
would cause a pair tokens to co-exist for the first time in any parse state,
where the two tokens can start with the same character and at least one of the
tokens can contain a character which is part of the grammar's separators.
 2017-02-27 22:54:38 -08:00
Max Brunsfeld
3c8e6f9987 Restructure parse state merging logic 
* Remove remnants of templatized remove_duplicate_states function
* Rename recovery_tokens function to get_compatible_tokens and augment it
  also compute pairs of tokens which could potentially be incompatible
 2017-02-26 12:23:48 -08:00
Max Brunsfeld
cefc57fe86 Move error cost comparisons into their own source file 2017-02-19 21:54:06 -08:00
Max Brunsfeld
32387400c6 Rework LR conflict resolution 
* Unify precedence/associativity-based resolution with the
  search for a whitelisted conflict
* Improve conflict error messages
 2016-11-18 13:50:55 -08:00
Max Brunsfeld
8d9c261e3a Don't include reduce actions for nonterminal lookaheads 2016-11-10 11:33:37 -08:00
Max Brunsfeld
22c550c9d6 Discard tokens after error detection to find the best repair 
* Use GLR stack-splitting to try all numbers of tokens to
  discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
  rather than the statically-computed 'in-progress symbols' list
 2016-05-11 13:49:43 -07:00
Max Brunsfeld
5b74813a5c Refine logic for which tokens to use in error recovery 2016-04-27 14:09:19 -07:00
Max Brunsfeld
dee1f697c1 Compute the set of variables that can begin with each terminal symbol 2016-02-25 21:51:52 -08:00
Max Brunsfeld
1543a6c7b0 Handle allocation failures when instantiating documents 2016-01-18 11:32:10 -08:00
Max Brunsfeld
b69e19c525 Add plain C API for compiling a JSON grammar 2016-01-10 13:44:22 -08:00
Max Brunsfeld
36870bfced Make Grammar a simple struct 2016-01-08 15:51:30 -08:00
Max Brunsfeld
f2e7058ad9 Support UTF16 directly 
This makes the API easier to use from javascript
 2015-12-28 13:53:22 -08:00
Max Brunsfeld
4ab4fd7346 Merge pull request #4 from maxbrunsfeld/mb-precedence-handling-within-tokens 
Make precedence more useful within tokens
 2015-11-01 21:50:53 -08:00
Max Brunsfeld
998ae533da Make completion_status() a method on LexItem 2015-10-30 16:48:37 -07:00
Max Brunsfeld
c8be143f65 🔥 get_metadata function 2015-10-30 16:22:25 -07:00
Max Brunsfeld
a26d7849ce Merge pull request #2 from robrix/get-symbol-names-from-documents 
Get symbol names from documents
 2015-10-29 10:51:13 -07:00
Rob Rix
d5f6581531 Add the runtime file to the project. 2015-10-29 12:46:51 -04:00
Max Brunsfeld
a8ead10d6f In lex error state, don't look for tokens that would match *any* line 2015-10-28 17:45:17 -07:00
Max Brunsfeld
db9966b57c Simplify lex item set transitions code 2015-10-11 22:51:37 -07:00
Max Brunsfeld
25791085c3 Normalize lexical grammar rules before constructing lex table 2015-10-11 16:56:00 -07:00
Max Brunsfeld
6d748a6714 Store parse actions' precedences as ranges, not sets 2015-10-05 16:05:19 -07:00
Max Brunsfeld
f01972c64e Reorganize ParseItemSet and LexItemSet 2015-10-05 15:09:06 -07:00
Max Brunsfeld
c4ef228397 Share common lookahead sets between parse item sets 2015-10-04 21:33:54 -07:00
Max Brunsfeld
ebc52f109d Merge branch 'flatten-rules-into-productions' 
This branch had diverged considerably, so merging it required changing a lot
of code.

Conflicts:
	project.gyp
	spec/compiler/build_tables/action_takes_precedence_spec.cc
	spec/compiler/build_tables/build_conflict_spec.cc
	spec/compiler/build_tables/build_parse_table_spec.cc
	spec/compiler/build_tables/first_symbols_spec.cc
	spec/compiler/build_tables/item_set_closure_spec.cc
	spec/compiler/build_tables/item_set_transitions_spec.cc
	spec/compiler/build_tables/rule_can_be_blank_spec.cc
	spec/compiler/helpers/containers.h
	spec/compiler/prepare_grammar/expand_repeats_spec.cc
	spec/compiler/prepare_grammar/extract_tokens_spec.cc
	src/compiler/build_tables/action_takes_precedence.h
	src/compiler/build_tables/build_parse_table.cc
	src/compiler/build_tables/first_symbols.cc
	src/compiler/build_tables/first_symbols.h
	src/compiler/build_tables/item_set_closure.cc
	src/compiler/build_tables/item_set_transitions.cc
	src/compiler/build_tables/parse_item.cc
	src/compiler/build_tables/parse_item.h
	src/compiler/build_tables/rule_can_be_blank.cc
	src/compiler/build_tables/rule_can_be_blank.h
	src/compiler/prepare_grammar/expand_repeats.cc
	src/compiler/prepare_grammar/extract_tokens.cc
	src/compiler/prepare_grammar/extract_tokens.h
	src/compiler/prepare_grammar/prepare_grammar.cc
	src/compiler/rules/built_in_symbols.cc
	src/compiler/rules/built_in_symbols.h
	src/compiler/syntax_grammar.cc
	src/compiler/syntax_grammar.h
 2015-10-02 23:46:39 -07:00