Best open-source AI codebases to track
A practical directory of widely used open-source AI projects across model frameworks, local inference, RAG, vector databases, agents, coding assistants, AI apps, creative generation, and MLOps. Links go to official repositories or project pages, with Pagish-authored summaries.
Model frameworks and core ML
Foundational libraries used to train, fine-tune, run, and evaluate modern AI systems.
These codebases sit underneath many products, papers, and applied AI stacks. They are worth tracking because changes here often become developer defaults.
Compiler, distributed training, inference, and model export improvements.
tensorflow/tensorflowTensorFlowLarge-scale machine learning, production deployment, mobile/edge ML, and mature enterprise workflows.Apache-2.0Serving, Lite, ecosystem maintenance, and model deployment tooling.
jax-ml/jaxJAXHigh-performance numerical computing, research models, differentiable programming, and accelerator-heavy workloads.Apache-2.0Compiler/runtime changes and ecosystem adoption in frontier research.
scikit-learn/scikit-learnscikit-learnClassical machine learning, preprocessing, clustering, model selection, and practical baseline models.BSD-3-ClauseAPI stability, explainable baselines, and integration with modern data workflows.
huggingface/transformersHugging Face TransformersUsing, fine-tuning, and shipping transformer models across language, vision, audio, and multimodal tasks.Apache-2.0New model architecture support and production-friendly inference paths.
huggingface/diffusersHugging Face DiffusersImage, video, and diffusion model pipelines with reusable schedulers and model components.Apache-2.0Video generation, controllability, memory optimization, and LoRA workflows.
huggingface/peftPEFTParameter-efficient fine-tuning methods such as LoRA and adapters for large models.Apache-2.0Adapter compatibility, quantized fine-tuning, and deployment-friendly merge paths.
UKPLab/sentence-transformersSentence TransformersEmbeddings, semantic search, retrieval, clustering, and RAG document matching.Apache-2.0Late-interaction retrieval, reranking, and multilingual embedding updates.
Local models and inference servers
Codebases that make it easier to run models locally, serve them at scale, or route requests across providers.
Inference determines whether AI systems are fast, affordable, private, and portable enough for real use.
Quantization formats, hardware backend support, and local multimodal inference.
ollama/ollamaOllamaDeveloper-friendly local model management with simple commands and a local API.MITModel library growth, tool compatibility, and desktop/local app integrations.
vllm-project/vllmvLLMHigh-throughput LLM serving, paged attention, OpenAI-compatible APIs, and production inference.Apache-2.0Parallelism, structured outputs, speculative decoding, and multimodal serving.
sgl-project/sglangSGLangFast serving and programming of language model applications with efficient runtime primitives.Apache-2.0Reasoning-model serving, constrained decoding, and multi-node performance.
huggingface/text-generation-inferenceText Generation InferenceProduction text-generation serving for Hugging Face models and hosted inference stacks.Apache-2.0Hardware support, batching, quantization, and model coverage.
BerriAI/litellmLiteLLMModel gateway, provider routing, OpenAI-compatible proxying, spend controls, and observability.MITEnterprise gateway features, provider support, and policy controls.
RAG, retrieval, and vector databases
Open-source systems for retrieval-augmented generation, embedding search, and AI-native data access.
Most useful AI products need grounded context, source retrieval, permissions, and searchable knowledge, not only a model call.
LangGraph integration, production patterns, and API simplification.
run-llama/llama_indexLlamaIndexData agents, RAG pipelines, indexes, connectors, and context engineering for enterprise knowledge.MITWorkflow APIs, agentic retrieval, and managed data integrations.
deepset-ai/haystackHaystackComposable RAG pipelines, search, question answering, and production retrieval systems.Apache-2.0Pipeline ergonomics, evaluation, and enterprise search connectors.
chroma-core/chromaChromaLocal and server vector storage for AI applications and lightweight RAG systems.Apache-2.0Persistence, scale, and developer experience.
qdrant/qdrantQdrantVector search service with filtering, payloads, and production retrieval workloads.Apache-2.0Hybrid search, distributed scaling, and cloud/on-prem portability.
milvus-io/milvusMilvusLarge-scale vector database workloads and embedding search infrastructure.Apache-2.0Operational simplicity, indexing performance, and ecosystem integrations.
weaviate/weaviateWeaviateAI-native vector database, semantic search, hybrid search, and RAG applications.BSD-3-ClauseQuery ergonomics, modules, and multi-tenancy features.
lancedb/lancedbLanceDBEmbedded and serverless vector search on Lance columnar data for multimodal AI apps.Apache-2.0Local-first workflows, multimodal retrieval, and lakehouse integration.
Agent frameworks and orchestration
Frameworks for tool-using agents, multi-agent systems, workflows, planning, and structured model interaction.
Agent frameworks are where model capability becomes repeatable work: tool calls, state, approvals, retries, memory, and traces.
Human-in-the-loop controls, persistence, and workflow templates.
microsoft/autogenMicrosoft AutoGenMulti-agent conversations, tool use, agent experiments, and autonomous workflow prototypes.MITAPI maturity, observability, and production deployment examples.
crewAIInc/crewAICrewAIRole-based agent teams, task delegation, and business-process agent prototypes.MITReliability controls, integrations, and reproducible workflow patterns.
microsoft/semantic-kernelSemantic KernelEnterprise app orchestration, planners, connectors, and model-provider abstraction.MIT.NET/Python parity, agent features, and enterprise templates.
stanfordnlp/dspyDSPyProgramming and optimizing language-model pipelines with measurable objectives.MITCompiler improvements, optimizers, and RAG/agent evaluation patterns.
pydantic/pydantic-aiPydantic AITyped Python agent apps with structured outputs, dependency injection, and testable model calls.MITProvider support, eval tooling, and production examples.
huggingface/smolagentssmolagentsSmall, readable agent framework for tool use, code agents, and quick experimentation.Apache-2.0Model/tool compatibility and examples that stay easy to audit.
Coding agents and developer copilots
Open-source projects that help developers edit code, automate repo tasks, or build AI-assisted development workflows.
Software work has tests, diffs, pull requests, and logs, so it is the best proving ground for practical agents.
Permission controls, MCP support, and provider compatibility.
continuedev/continueContinueOpen-source AI code assistant with autocomplete, chat, edit workflows, and custom models.Apache-2.0Local model support, IDE coverage, and team configuration.
Aider-AI/aiderAiderTerminal pair programming, repo-aware edits, and git-friendly coding assistance.Apache-2.0Model support, benchmark movement, and large-codebase behavior.
sst/opencodeOpenCodeTerminal-native AI coding agent workflows with model choice and developer control.MITAgent reliability, editor integration, and open-source community growth.
All-Hands-AI/OpenHandsOpenHandsAutonomous software engineering agents, issue resolution, and sandboxed coding workflows.MITBenchmark results, sandbox security, and reviewable task traces.
gpt-engineer-org/gpt-engineerGPT EngineerPrompt-to-code experiments and agent-assisted project generation.MITScope control, maintainability, and integration with modern coding agents.
Open AI apps and workflow builders
End-user and team-facing open-source apps for chat, agents, workflow automation, and model access.
These projects are often the fastest way to understand what people actually deploy around open models and enterprise workflows.
Workflow maturity, self-hosting, and enterprise governance features.
open-webui/open-webuiOpen WebUISelf-hosted ChatGPT-like interface for local and hosted models, especially Ollama users.BSD-3-ClauseModel/provider integrations, admin controls, and plugin ecosystem.
danny-avila/LibreChatLibreChatSelf-hosted multi-provider AI chat with assistants, agents, plugins, and team controls.MITEnterprise features, provider coverage, and deployment templates.
Mintplex-Labs/anything-llmAnythingLLMPrivate AI knowledge workspaces, document chat, RAG, and local/hosted model use.MITDesktop/server parity, connectors, and workspace governance.
FlowiseAI/FlowiseFlowiseVisual LLM workflow builder for chains, agents, RAG, and prototype-to-app workflows.Apache-2.0Production deployment patterns and visual workflow maintainability.
n8n-io/n8nn8nWorkflow automation with AI nodes, integrations, and self-hosted business automation.Fair-code/Sustainable UseAI workflow templates, licensing boundaries, and connector depth.
openclaw/openclawOpenClawPersonal AI assistant workflows that connect local devices, model providers, tools, messaging channels, and companion apps through a gateway.MITChannel integrations, local-control security, plugin maturity, and how well the gateway model works for single-operator assistants.
Creative AI and multimodal generation
Open-source interfaces and tooling for image, video, audio, speech, and multimodal generation workflows.
Creative AI adoption depends on controllability, repeatable workflows, extension ecosystems, and rights-aware production habits.
Custom node ecosystem, video workflows, and model support.
AUTOMATIC1111/stable-diffusion-webuiAUTOMATIC1111 Stable Diffusion WebUIStable Diffusion web interface, extensions, image generation, and creative experimentation.AGPL-3.0Extension compatibility, model support, and maintenance cadence.
invoke-ai/InvokeAIInvokeAIProfessional image-generation workflows, canvas editing, queues, and team-friendly creative AI.Apache-2.0Workflow UX, deployment options, and commercial production features.
ggml-org/whisper.cppWhisper.cppFast local speech recognition and transcription across CPUs, GPUs, and edge devices.MITHardware acceleration, streaming, and mobile deployment.
coqui-ai/TTSCoqui TTSText-to-speech research, voice synthesis, and speech model experimentation.MPL-2.0Maintenance state, model availability, and licensing clarity for voices.
hpcaitech/Open-SoraOpen-SoraOpen video generation research and reproducible text-to-video training experiments.Apache-2.0Model quality, training recipes, and practical inference requirements.
MLOps, evaluation, and observability
Open-source systems for experiments, deployment, monitoring, evaluation, and operational reliability.
AI work becomes useful when teams can evaluate, ship, observe, govern, and improve systems after the demo.
GenAI evaluation features, tracing, and registry integrations.
ray-project/rayRayDistributed Python, training, tuning, serving, and scalable AI workloads.Apache-2.0Ray Serve, LLM serving patterns, and cluster reliability.
kubeflow/kubeflowKubeflowKubernetes-native ML workflows, pipelines, training operators, and platform engineering.Apache-2.0Install simplicity, pipeline experience, and cloud portability.
bentoml/BentoMLBentoMLPackaging, serving, and deploying models and inference APIs.Apache-2.0LLM serving ergonomics, deployment targets, and scale patterns.
langfuse/langfuseLangfuseLLM tracing, prompt management, evaluation, and observability for AI apps.MITSelf-hosting, eval workflows, and integration coverage.
Arize-ai/phoenixArize PhoenixLLM tracing, RAG evaluation, observability, and dataset inspection.Elastic-2.0OpenTelemetry support, eval UX, and production integrations.
explodinggradients/ragasRagasRAG evaluation, synthetic testsets, answer quality metrics, and retrieval diagnostics.Apache-2.0Metric quality, testset generation, and evaluation reproducibility.
confident-ai/deepevalDeepEvalUnit-test-like evaluation for LLM apps, RAG, agents, and regression testing.Apache-2.0CI integration, agent evals, and metric reliability.