Docker Compose environments for demonstrating modern data platform architectures using Kafka, Flink, Spark, Iceberg, Pinot + Kpow & Flex by Factor House

Tools for processing sql queries metadata.

Covid19 and Iowa Liquor Sales analysis at BigQuery using dbt, Airflow, Marquez, Google Cloud and other modern data stack tools

NextGen DataMotion Lineage

Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.

Testing a Docker deployment of Marquez and OpenLineage

This Project aims to enhance Spark Lineage Generation in Open Metadata by creating Lineage edges from Table-to-Table with Column level lineage as well as Container to Table Lineage.

Exploring how to surface dlt pipeline metadata via Airflow's OpenLineage integration.