☁️ Build multimodal AI applications with cloud-native stack

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Turns Data and AI algorithms into production-ready web applications in no time.

Transform data, train models, and run SQL with marimo — feels like a next-gen reactive notebook, stored as Git-friendly reproducible Python. Deploy as scripts, pipelines, endpoints, and apps. All from an AI-native editor (or your own).

Always know what to expect from your data.

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

Install Jenkins, configure Docker as slave, set up cicd, deploy applications to k8s using Argo CD in GitOps way.

🧙 Build, run, and manage data pipelines for integrating and transforming data.

📚 Parameterize, execute, and analyze notebooks

cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，mlops算法链路全流程，算力租赁平台，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU虚拟化，边缘计算，标注平台自动化标注，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库，AI模型市场，支持国产cpu/gpu/npu 昇腾生态，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

Machine Learning Pipelines for Kubeflow

Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.

Blender addons that improve the game development workflow between Blender and Unreal.

Python library for creating data pipelines with chain functional programming

HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Data intensive science for everyone.