The maeel programming language
-
Updated
Jul 9, 2024 - Rust
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
The maeel programming language
Contrastive-LSH Embedding and Tokenization Technique for Multivariate Time Series Classification
The greynir.is Icelandic natural language processing API and website.
A high-performance axum API for serving Hugging Face's Tokenizers
📐 GPT token estimation and context size utilities without a full tokenizer
C language lexer & parser & virtual interpreter from scratch in Rust
KPT: Kannada Pre-trained Transformer
⛄ Possibly the smallest Lua compiler ever
Testing Automatic Text Summarization
Parser Building Toolkit for JavaScript