Airee lets you use Apache Airflow to create data pipelines without need of manage the underlying infrastructure for scalability, availability, and security. It is integrated with security services to help provide you with fast and secure access to your data.
Airee covers the entire process, from design and installation, to Apache Airflow implementation, testing and debugging:
- Airee deploys Apache Airflow instance on Google Cloud Platform or on-premises infrastructure using Kubernetes.
- Airee offers predefined environment set-ups according to number of running tasks.
- Airee supports secret management using GCP Secret Manager and TLS communication.
- Airee’s strong points are continuous integration CI and continuous delivery CD processes supported by GitHub Actions, Terraform and Flux.
- Airee provides a setup of GitHub Actions runner deployed on Google Kubernetes Engine
- Airee provides local Apache Airflow environment for development purposes.
- Airee is transparent, users are granted access to GitHub code for infrastructure, application and DAGs templates.
- Airee supports various Apache Airflow executors including CeleryExecutor, KubernetesExecutor and CeleryKubernetesExecutor.
- Airee supports KEDA autoscaller.
- Airee takes into account cloud cost-effectiveness and provides option to pause infrastructure if it is unused.
- Airee is an open-sourced project, users can develop customization suited for their business case.
Airee Use Cases:
- First time Apache Airflow user.
If you are looking for orchestration tool and you found Apache Airflow, then Airee helps you with deployment of infrastructure in cloud or on premise on Kubernetes Cluster, it covers configuration and secure connections between components.
It is cost effective and open to customization. Airee will spin up your Apache Airflow instance in the cloud and manage changes with automated CI/CD.
- Heavy user of Apache Airflow with need for custom features.
If you are a heavy Apache Airflow user, you have your own plugins and operators, you need tailor made deployment with company policies integration, you can benefit from Airee with CI/CD features and ready to use deployment. As a team maintaining Airee we can also support you with development of custom features – just create an issue or a PR in this repository for general requests or in particular repository for detailed changes.
- Running Apache Airflow at a scale.
When you work in a data driven organization and you need more and more reports and analysis to be delivered each day at exact hour, you can use Airee as a platform for provisioning Apache Airflow instances in a unified way.
Airee delivers ready to use Terraform recipes for Kubernetes Custer deployment and prepared CI/CD definitions for GitHub Action.
You can transform our containerized Controller into a REST API that spins up Apache Airflow instances on demand.
They were very impressive with their thoroughness of research and approach to kicking off the project.
Head of Product Development, Sportside
Their commitment, knowledge, and good communication resulted in high performance and a comfortable work atmosphere.
CEO, Macmos Stream
Our technology tool stack
- Analytical Databases: Big Query, Redshift, Synapse
- ETL: Databricks, DataFlow, DataPrep
- Scalable Compute Engines: GKE, AKS, EC2, DataProc
- Process Orchestration: Apache Airflow / Composer, Bat
- Platform Deployment & Scaling: terraform, custom tools
- Support for all Hadoop distributions: Cloudera, Hortonworks, MapR
- Hadoop tools: hdfs, hive, pig, spark, flink
- No SQL Databases: Cassandra MongoDB, Hbase, Phoenix
- Process Automation: oozie, Apache Airflow
Get in touch with us
Contact us to see how we can help you.
We’ll get back to you within 4 hours on working days (Mon – Fri, 9am – 5pm).