Blog
Michal Milosz

Latest blog posts by Michal Milosz

Latest blog posts

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
No blog posts found.
Data Engineering
10
min read

The Power of Orchestration: Managing Complex Workflows in Databricks

Master Databricks workflow orchestration – automate tasks, integrate Airflow & ADF, and optimize pipelines. Boost efficiency with dynamic scheduling!
Read more
Data Engineering
11
min read

Azure Data Factory or Apache Airflow: Which Orchestration Tool Reigns Supreme?

Compare Azure Data Factory vs Apache Airflow: key features, strengths, and limitations for data orchestration. Choose the best tool for your workflow needs.
Read more
Databricks
9
min read

Managing Large Data Sets in Databricks

Optimize large datasets in Databricks with Partitioning, Z-Ordering, Auto Optimize, Delta Lake Vacuum, Caching, and Cost Monitoring for better performance.
Read more
Data Engineering
12
min read

Configuring the Celery Kubernetes

Configure the Celery Kubernetes Executor in Airflow 2.0 to combine Celery's scalability with Kubernetes' resource optimization for efficient workflows.
Read more
Data Engineering
7
min read

The Future of Data Engineering – Trends to Watch in 2025

Read more
Data Engineering
6
min read

Data Warehouse vs. Data Lake vs Lakehouse: A Comprehensive Comparison of Data Management Approaches

Read more
DevOps
min read

How to spin up to 1000 parallel Airflow 2.0 tasks in 5 minutes from nothing with the CeleryExecutor and Kubernetes

Read more
Data Science
min read

Stream Processing vs. Batch Processing - A Practical Guide to Data Processing

Explore stream vs batch processing, their benefits, use cases, and technologies. Learn how to choose the right data strategy for your business needs.
Read more
Data Science
min read

Microservices in Data Engineering: How to Break a Monolith into Smaller Parts

Discover how microservices revolutionize data engineering by replacing monolithic systems, boosting scalability, flexibility, and deployment efficiency.
Read more
Data Science
min read

From Excel to Data Lake: The Evolution of Data Storage in a Modern Organization

Explore the shift from Excel to data lakes, modern storage solutions, and best practices for scalable, efficient, and innovative data management strategies.
Read more