1. Data pipeline & ETL
  2. Open Source Tools for AI-Projects

Open Source Tools for AI-Projects

Displaying 11 - 20 of 21
prefect by PrefectHQ


The easiest way to automate your data
azkaban by azkaban


Azkaban workflow manager.
setl by SETL-Framework


A simple Spark-powered ETL framework that just works 🍺
luigi by spotify


Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
couler by couler-proj


Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
edge-ml by edge-ml
gokart by m3dev


Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
genie by Netflix


Distributed Big Data Orchestration Service
basin by basin-etl


Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser
pipelinex by Minyus


PipelineX: Python package to build ML pipelines for experimentation with Kedro, MLflow, and more