constrained-decoding

Query language for blending SQL and local language models across structured + unstructured data, with type constraints.

Reproduction Package for the paper "Type-Constrained Code Generation with Language Models" [PLDI 2025]

Constrained Decoding of Diffusion LLMs with Context-Free Grammars.

For our ICRA 2025 paper 🏆 "SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models" by Yi Wu, Zikang Xiong, Yiran Hu, Shreyash Iyengar, Nan Jiang, Aniket Bera, Lin Tan, and Suresh Jagannathan. (🏆 Best Paper Award Finalist!)

📐 Just-in-time logic enforcement for network data generation with LLMs

A guide to structured generation using constrained decoding

Code for paper "Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation" EMNLP 2021 and "Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation" arXiv 2020

Local inference you can prove. A single-binary GGUF engine in C: serves, scores, and trains LoRA through the quantized weights it deploys, byte-reproducibly. Tool calls survive the token limit; every run replays as a signed receipt. CPU/CUDA/Metal, OpenAI- and Anthropic-compatible.

A tool-calling layer, not a language model — your schemas in, validated calls out, at 48M parameters. Malformed JSON, invented parameter names, and undeclared tools are structurally unreachable on any catalog. Adapt it to your own catalog; 11 negative results included.

Constraint-driven code generation that transforms AI from probabilistic guessing into controlled search through valid program spaces.

[Pytorch] Efficient tokenization for recommendations and generative retrieval. Inspired by STATIC decoding from "Vectorizing the Trie"

Deterministic, integer-only SLM inference runtime — no floating point on the inference path, so the same model and prompt produce bit-identical output across CPUs, GPUs, compilers, and vendors.

Speculative grammar backtracking algorithm for LLM decoding conforming to some lark context-free grammar (CFG)

Curated list of the open System One ecosystem: open models, independent benchmarks, calibration and constrained-decoding tooling.

Every language before it was built for humans to read and learn. LOVA is the first built for AI: programs are integer sequences — 64 one-byte operators, structured faults with repair hints, declared effects, budgets, lineage, a Rust VM at 11M steps/s, an MCP server. Open source: extend, modify or rebuild it to make AI faster.

PEG grammars as LLM output constraints. TypeScript-native parser generator with parametric rules.

🧠 Visualize token-by-token sampling with chat templates, nucleus filtering, constrained decoding, and attribution heatmaps in one local app ⚡

Artifact-backed evaluation of how structured-output contracts change LLM semantics, validity, and execution outcomes.

🤖 AI-powered function calling engine using a LLM to convert natural language prompts into validated JSON function calls.

VibeDrift - Run any LLM on your own hardware. Bypass the VRAM wall with CPU/RAM inference, MOE expert offloading, and 4-bit quantization. No Cloud, no Subscription.