Commit graph

13 commits

Author SHA1 Message Date
Amaan Qureshi
a69367f739 feat: add API for editing points and ranges 2025-09-19 17:40:26 -04:00
Amaan Qureshi
8943983df6 feat!: properly handle UTF-16 endianness encoding 2024-10-05 21:12:48 -04:00
Max Brunsfeld
dfc6d93bae Rename wasm C sources to avoid conflicts with standard wasm headers
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 13:08:30 -08:00
Max Brunsfeld
98ccfcffb0 Provide minimal C/C++ std library to wasm external scanners 2022-11-15 17:14:33 -08:00
Max Brunsfeld
3f1a7f9cd4 Start work on ability to load wasm languages from native lib, via wasmtime 2022-11-15 17:14:33 -08:00
Mingkai Dong
e742186c25 Allow to change the allocator dynamically 2021-12-17 20:16:20 +08:00
Matthew Krupcale
ee9a3c0ebb lib: remove utf8proc dependency (#436)
* Remove dependency on utf8proc

This removes the only external dependency on utf8proc for UTF-8 decoding. It does so by implementing its own UTF-8 decoder. This decoder is both faster and has a simpler API.

 * .gitmodules: remove utf8proc submodule
 * docs/section-2-using-parsers.md: remove requirement for utf8proc submodule
 * docs/section-6-contributing.md: likewise
 * lib/Cargo.toml: remove utf8proc subdirectory package include
 * lib/README.md: remove utf8proc subdirectory description
 * lib/binding_rust/build.rs: remove utf8proc compiler include directory
 * lib/src/lexer.c: remove utf8proc dependencies and types
 * lib/src/lib.c: remove utf8proc dependency
 * lib/src/unicode.h: define types for Unicode decoders
 * lib/src/utf16.{c,h}: implement more readable UTF-16 decoder
 * lib/src/utf8.{c,h}: implement fast UTF-8 decoder
 * lib/utf8proc: remove utf8proc submodule directory
 * script/build-lib: remove utf8proc compiler include directory
 * script/build-wasm: likewise

* Optimize ts_lexer__get_lookahead.

Try to favor non-failure code path and assign lookahead values directly to lexer

 * lib/src/lexer.c: optimize for non-failure code path

* Fix some compiler errors

 * lib/src/lexer.c: cast from signed to unsigned for decode_next result
 * lib/src/utf16.c: fix non-constant initializers for older compilers

* Remove some missed remnants of utf8proc

 * docs/section-2-using-parsers.md: only two include paths necessary now
 * lib/src/lib.c: no need to define UTF8PROC_STATIC

* Use ICU's utf8 and utf16 decoding routines

* Remove unnecessary casts when calling icu macros

* Check buffer length before attempting to decode a unicode character

* Use new unicode function when parsing Queries

Co-Authored-By: Matthew Krupcale <mkrupcale@matthewkrupcale.com>

* Mark libicu files as vendored for GitHub's stats
2019-10-14 11:18:39 -07:00
Max Brunsfeld
fe7c74e7aa Start work on an API for querying trees 2019-09-10 20:53:57 -07:00
Joshua Warner
6f71b8840b Fix warning about undefined snprintf 2019-03-08 21:22:13 -08:00
Max Brunsfeld
a85ed086cc Define UTF8PROC_STATIC macro in the source
This way, there's one less flag you have to pass when building the
library on windows.
2019-02-06 10:42:29 -08:00
Vicent Marti
9a951c859d lib: Fix importing fdopen in parser.c 2019-02-06 10:10:38 +01:00
Max Brunsfeld
d192eda9cf Remove stray word 'runtime' from comment 2019-01-30 21:43:44 -08:00
Max Brunsfeld
0f2347b318 Just call the C lib 'the library' everywhere, don't call it a 'runtime' 2019-01-10 15:22:39 -08:00
Renamed from lib/src/runtime.c (Browse further)