Infrastructure

LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Port of OpenAI's Whisper model in C/C++

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A generative speech model for daily dialogue.

Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

Find secrets with Gitleaks 🔑

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

Faster Whisper transcription with CTranslate2

A list of free LLM inference resources accessible via API.

End-to-end, code-first tutorials for building production-grade GenAI agents. From prototype to enterprise deployment.

High-performance In-browser LLM Inference Engine

The absolute trainer to light up AI agents.

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database​.

An orchestration platform for the development, production, and observation of data assets.

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Official inference library for Mistral models

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Local AI anywhere, for everyone — LLM inference, chat UI, voice, agents, workflows, RAG, and image generation. No cloud, no subscriptions.

A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.