Commit graph

11 commits

Author SHA1 Message Date
Matthew Krupcale
ee9a3c0ebb lib: remove utf8proc dependency (#436)
* Remove dependency on utf8proc

This removes the only external dependency on utf8proc for UTF-8 decoding. It does so by implementing its own UTF-8 decoder. This decoder is both faster and has a simpler API.

 * .gitmodules: remove utf8proc submodule
 * docs/section-2-using-parsers.md: remove requirement for utf8proc submodule
 * docs/section-6-contributing.md: likewise
 * lib/Cargo.toml: remove utf8proc subdirectory package include
 * lib/README.md: remove utf8proc subdirectory description
 * lib/binding_rust/build.rs: remove utf8proc compiler include directory
 * lib/src/lexer.c: remove utf8proc dependencies and types
 * lib/src/lib.c: remove utf8proc dependency
 * lib/src/unicode.h: define types for Unicode decoders
 * lib/src/utf16.{c,h}: implement more readable UTF-16 decoder
 * lib/src/utf8.{c,h}: implement fast UTF-8 decoder
 * lib/utf8proc: remove utf8proc submodule directory
 * script/build-lib: remove utf8proc compiler include directory
 * script/build-wasm: likewise

* Optimize ts_lexer__get_lookahead.

Try to favor non-failure code path and assign lookahead values directly to lexer

 * lib/src/lexer.c: optimize for non-failure code path

* Fix some compiler errors

 * lib/src/lexer.c: cast from signed to unsigned for decode_next result
 * lib/src/utf16.c: fix non-constant initializers for older compilers

* Remove some missed remnants of utf8proc

 * docs/section-2-using-parsers.md: only two include paths necessary now
 * lib/src/lib.c: no need to define UTF8PROC_STATIC

* Use ICU's utf8 and utf16 decoding routines

* Remove unnecessary casts when calling icu macros

* Check buffer length before attempting to decode a unicode character

* Use new unicode function when parsing Queries

Co-Authored-By: Matthew Krupcale <mkrupcale@matthewkrupcale.com>

* Mark libicu files as vendored for GitHub's stats
2019-10-14 11:18:39 -07:00
Max Brunsfeld
47607cecf4 Reorganize repo, add rust CLI and binding code, 2019-01-04 17:31:49 -08:00
Max Brunsfeld
6e4115548c Initial commit 2018-05-17 14:30:03 -07:00
Max Brunsfeld
bf4b8bf55b Use my fork of crypto-algorithms 2017-07-10 14:29:14 -07:00
Max Brunsfeld
fcffd4b732 Add test for an example found during fuzzing 2017-06-30 21:55:50 -07:00
Max Brunsfeld
b69e19c525 Add plain C API for compiling a JSON grammar 2016-01-10 13:44:22 -08:00
Max Brunsfeld
3bec739202 Replace vendored copy of utf8proc w/ submodule reference
The code is now on GitHub
2015-12-27 13:26:56 -08:00
Max Brunsfeld
67e008d15d Generate makefile with gyp
- Add gyp submodule
- Add script to generate makefile with gyp
- Switch travis build to do this
2014-03-03 13:20:44 -08:00
Max Brunsfeld
d027aa5af6 Switch specs to use bandit instead of igloo 2013-12-28 16:55:51 -08:00
Max Brunsfeld
e480cf538d Move igloo submodule out of specs directory 2013-12-15 23:25:31 -08:00
Max Brunsfeld
84c5bceb81 Initial commit 2013-11-05 22:15:19 -08:00