Optimizing Apache Spark Apache Spark is a powerful tool for data processing, which allows for orders of magnitude improvements in execution times compared to Hadoop’s...
Azure SQL authentication with AD When using Azure SQL instance it may be tempting to just create database user and use simple sql authentication –...
First things first In data science and python world there is a very well known package pandas. But… what is pandas? Pandas provide essential data...
Databricks testing with GitHub Actions To those who inspired it and will never read it. And for Marcin! Introduction Databricks Connect (more info here) provides a good...
From out of the box to totally custom In project, where I’m partially responsible for integration airflow with other components, was assigned to me simple...