Reproducible Data Science that Scales! Pachyderm lets you deploy and manage multi-stage, language-agnostic data pipelines while maintaining complete reproducibility and provenance. 6 is out and ready for production use! Version control for data. Pachyderm version controls data, similar to what Git does with code. You can track the state of your data over time, backtest models on historical data, share data with teammates, and revert to previous states of data.

Pachyderm lets you store and analyze your data using containers. Pachyderm builds on cutting edge tools in the container ecosystem. To provide a complete containerized data analytics package. Pipelines of jobs are defined as Docker containers, so you can use any language and any libraries you want. If it fits in a container, you can use it for data analysis.

Getting Your Data into Pachyderm.