Skip to content
#

mtp

Star

Here are 292 public repositories matching this topic...

Language: All
Filter by language
All 292 Python 103 Shell 32 Swift 22 C# 18 C++ 18 C 14 Rust 14 JavaScript 11 HTML 7 PowerShell 7
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

youssofal / MTPLX

Star 2.6k

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

macos metal mtp mlx inference-engine apple-silicon local-llm llm-inference local-ai qwen speculative-decoding openai-compatible claude-code qwen3-next anthropic-compatible native-mtp mtplx qwen3-8 qwen3-8-flash-next flash-next
  • Updated Oct 10, 2026
  • Python

SergiioB / intel-arc-pro-b70-inference-cookbook

Star 175

Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!

moe mtp sycl intel-gpu intel-arc llama-cpp vllm llm-inference local-ai speculative-decoding battlemage intel-arc-b70
  • Updated Oct 7, 2026
  • Python

JakeATX / llamAmpere

Star 168

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

cuda quantization mtp ampere llm long-context llama-cpp ggml local-llm llm-inference qwen gguf speculative-decoding kv-cache-quantization qwen3 consumer-gpu multi-token-prediction rtx-3090 turboquant rtx-3090-ti
  • Updated Oct 12, 2026
  • C++

DominikBucko / qwen38-flash-next-2x3090

Star 114

Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).

moe quantization mtp multi-gpu dual-gpu rtx3090 int4 fp8 vllm local-llm llm-inference qwen speculative-decoding rtx4090 rtx-3090 cpu-offloading qwen3-8 qwen38 qwen3-8-flash-next 256k-context
  • Updated Oct 2, 2026
  • Python

Etamus / NeveAI

Sponsor
Star 65

NeveAI é uma plataforma de IA local privacy-first, desenvolvida para oferecer uma experiência de alta performance na execução de LLMs, reduzindo a dependência de grandes plataformas, assinaturas caras e APIs externas e servindo como uma alternativa offline, privada e independente.

ui image-generation node-js mtp web-search privacy-first pyodide llm llms llama-cpp agentic-workflow mcp-tools neveai
  • Updated Oct 9, 2026
  • Python

Add this topic to your repo

To associate your repository with the mtp topic, visit your repo's landing page and select "manage topics."

Learn more