Knowledge Sharing (KS) - Benchmark of data processing engines

Table of Content (ToC)

Overview

This project aims at benchmarking a few data processing engines (e.g., DuckDB, Spark, Polars, Daft).

Inspiration: TODO MVC

Even though the members of the GitHub organization may be employed by some companies, they speak on their personal behalf and do not represent these companies.

References

• https://todomvc.com/
• Data Engineering Helpers - Knowledge Sharing - Cheat sheets
  • Data Engineering Helpers - Knowledge Sharing - Python
  • Data Engineering Helpers - Knowledge Sharing - Spark
  • Data Engineering Helpers - Knowledge Sharing - DuckDB
  • Data Engineering Helpers - Knowledge Sharing - SQLMesh
  • Data Engineering Helpers - Knowledge Sharing - dbt
• Material for the Data platform - Architecture principles
• Material for the Data platform - Modern Data Stack (MDS) in a box
• Material for the Data platform - Data life cycle
• Material for the Data platform - Data contracts
• Material for the Data platform - Metadata
• Material for the Data platform - Data quality