All Meta AI items under Companies on AIFIRST News, newest first. 18 items.
A developer found the official llama.cpp build is not optimized for AMD Strix Halo (gfx1150) hardware and shared an alternative setup guide for achieving the highest possible throughput on that chipset. The post notes roughly 90% of Strix Halo users are still running the unoptimized stock build.
Mark Zuckerberg has confirmed on X that open weights for Meta's Muse Spark will be released soon. The announcement is notable because Muse Spark is reported to match Anthropic Fable 5 performance. Community reaction is mixed, with some users waiting for Llama 5 instead due to size constraints.
A user running Claude Code against a self-hosted LiteLLM and llama.cpp backend encountered a prompt injection attempt. Claude Code identified and flagged the injected instruction rather than executing it. The post is a practical example of prompt injection risks in agentic AI pipelines and how detection can work in practice.
Vision support has been merged into DeepSeek-V4-Flash-Vision-Exp, with Unsloth GGUF quantized variants now available on Hugging Face. The update enables local users to run the vision-capable DeepSeek Flash model through llama.cpp-compatible tools. Multiple quantization levels are available for different hardware configurations.
A local inference enthusiast documented achieving 280 tokens per second on Qwen3.8 27B using two AMD Radeon RX 9700 GPUs with a 940k token KV cache. The writeup covers driver setup, ROCm configuration, and the specific llama.cpp flags that unlocked the result. Dual R9700 performance figures are scarce, making this a useful reference for the AMD open-source inference path.
In addition to the Hugging Face platform acquisition, Nvidia would effectively acquire the llama.cpp project and its core development team, which joined Hugging Face in February. llama.cpp is the dominant open-source runtime for running local LLMs on consumer hardware. This concentration raises questions about whether the project will remain hardware-neutral and freely licensed going forward.
llama.cpp has merged support for Qwen 3.8 Flash Next, the new efficient Qwen model variant. GGUF quantizations are now available, with one user reporting 55 tokens per second on a four-GPU RTX 3090 setup using Q4 quantization. The merge makes the model accessible through the standard llama.cpp toolchain without additional patching.
audio.cpp version 0.7 has been released, expanding the project to 62 audio model families with over 85 model variants available for local inference. New in this release is an Arena UI that lets users compare models side by side locally without sending audio to cloud services, and support for recent MiniMax audio models. audio.cpp follows a similar architecture to llama.cpp but focuses entirely on audio generation, voice, and speech models.
GLM-4.5-Air, a 106B mixture-of-experts model with only 12B active parameters, now supports Multi-Token Prediction in llama.cpp for a meaningful inference speed boost. The update makes it more practical for users who want a large MoE without requiring multiple high-end GPUs. MTP support is enabled via a configuration flag in the updated build.
The ConvRot quantization method, which outperforms standard UDQ8_K_XL in accuracy benchmarks, has been integrated into the llama-cpp-turboquant fork. ConvRot applies a learned rotation to model weights before quantization, reducing the accuracy loss from low-bit formats. The fork is available on GitHub for users who want better quantization quality beyond standard GGUF methods.
A developer ran a 3-day benchmark of Inco AI's DFlash 2 speculative decoding PR in llama.cpp against plain decoding, MTP, n-gram lookup drafters, and the previous DFlash 1 on Qwen 3.8 27B. DFlash 2 introduces a trained drafter specifically for Qwen 3.8 27B and shows meaningful throughput improvements over all comparison methods.
A developer released a llama.cpp fork with HIP optimizations targeting AMD GFX906-class GPUs including the Mi50, Mi60, Radeon VII, and GCN HIP cards. These cards have minimal support in the mainline llama.cpp codebase, leaving many AMD users without an optimized inference path.
nRouter SDK puts a single API key and one interface in front of six cloud model providers: Alibaba US, OpenAI, AWS Bedrock, Azure Foundry, Google Vertex AI, and Anthropic. It speaks both the OpenAI and Anthropic wire formats natively and covers chat completions, embeddings, images, text to speech, speech to text, and video. Branded packages ship for Python, JavaScript, Java, Rust, and R, with OpenAI-compatible paths for Go, Ruby, PHP, and C#. Framework integrations include LangChain, LlamaIndex, Vercel AI SDK, CrewAI, and AutoGen.
llama.cpp is a C/C++ implementation of large language model and vision language model inference, built on the ggml tensor library and MIT licensed. It ships a command line interface, an OpenAI-compatible API server, and a web chat UI, installable as an app, via Docker, from pre-built binaries, or from source. Backends cover Apple Silicon with ARM NEON, Accelerate and Metal, x86 with AVX through AVX512 and AMX, NVIDIA CUDA, AMD HIP, Vulkan and SYCL, plus hybrid CPU and GPU inference. The repository showed 124.9k stars and 22.0k forks with 778 open issues and roughly 1.4k open pull requests, and quantization spans 1.5-bit to 8-bit so output quality depends on the level chosen.
Vomit is a small open-source tool that intercepts Claude agent mode's intermediate reasoning output and pipes it through a local LLM to produce more readable English summaries. It runs fully locally with no telemetry, supports Llama.app and Ollama backends, and can be used either as an integrated hook replacing Claude's displayed output or as a standalone session monitor via the 'vomit tail' command. The tool targets the verbosity of Claude's token output during agentic tasks.
Shoehorn is a locally run, browser-based tool that quantizes language models with hardware-aware mixed precision, assigning a precision level per tensor rather than applying a preset and targeting about 99.99% of available memory. It scans popular Hugging Face models to work out what fits a given machine and writes standard GGUF v3 files for llama.cpp. It supports macOS on Apple Silicon, Linux on x86-64, and Windows on x86-64. The quantizer core is written from scratch in Rust.
LLaMA-Factory provides a unified web UI + CLI for fine-tuning LLMs and VLMs without writing custom training code. Supports LLaMA, Mistral, Qwen, DeepSeek, Gemma, Phi, Yi, and 90+ others. Positioned as a no-code altern...
LlamaIndex's Jerry Liu is open sourcing ParseBench - a benchmark specifically measuring document parsing quality for AI agent workflows. The argument: current OCR/parsing benchmarks don't reflect real agentic use case...
AIFIRST 
