Latest blog posts by Michal Milosz

Learn how adopting a “shift left” mindset with unit-tested, modular PySpark code, local testing, and CI/CD automation can cut cloud costs, prevent production bugs, and make your data pipelines behave like real engineered software.

Learn how to choose between declarative Delta Live Tables and imperative Databricks Workflows to design scalable, cost‑efficient data architectures that match your team’s culture and operational needs in the modern Azure and Databricks ecosystem.

Automating Databricks deployments with DAB templates and CI/CD tools ensures fast, error-free pipelines - read to learn how.

Discover how Unity Catalog and Volumes in Databricks revolutionize data governance by enabling secure, centralized management and effortless discovery of both structured and unstructured data across your Lakehouse.

Discover the strengths and uses of three key data architectures - find out how they work and which one fits your needs best

Master Databricks workflow orchestration – automate tasks, integrate Airflow & ADF, and optimize pipelines. Boost efficiency with dynamic scheduling!

Compare Azure Data Factory vs Apache Airflow: key features, strengths, and limitations for data orchestration. Choose the best tool for your workflow needs.

Optimize large datasets in Databricks with Partitioning, Z-Ordering, Auto Optimize, Delta Lake Vacuum, Caching, and Cost Monitoring for better performance.

Configure the Celery Kubernetes Executor in Airflow 2.0 to combine Celery's scalability with Kubernetes' resource optimization for efficient workflows.

Discover the top data engineering trends for 2025, including AI-driven automation, Lakehouse architecture, serverless and edge computing, and sustainable data practices. Embracing these innovations will help organisations optimise data operations and stay ahead in a rapidly changing landscape.