Success story

The customer’s objective was to overcome data silos and construct a unified Datalake that serves as a reliable and authoritative source of data for campaigns and planning. The desired solution needed to be cost-effective, secure, and dependable, while adhering to data stewardship standards. The client had high expectations for seamless integration with multiple third-party data vendors and the implementation of scalable, fully-managed ETL solutions available through the Google Cloud Platform.

In order to meet these objectives, we devised a comprehensive set of integration processes that guarantee efficient data ingestion, transformation, and storage. Our approach involved the creation and scheduling of more than 100 pipelines in Composer, leveraging the power of DataProc clusters, Google Cloud Storage buckets, and BigQuery. We seamlessly migrated historical data bundles using specialized Data Transfer tools. To ensure the utmost data security, we implemented Secret Manager, while also employing Logging to diligently monitor the processes.

The seamless migration to the cloud facilitated the efficient processing of data on a massive scale, surpassing a petabyte. Over the course of 6 months, we accomplished the successful migration of roughly 1.5PB of historical data, restructured numerous ingestion processes, and orchestrated streamlined pipelines on the Google Cloud Platform. Consequently, data storage and computation expenses decreased by approximately 30%. The punctual and error-free delivery of data empowers a multitude of internal applications and significantly benefits more than 50 brands operating in over 100 markets.

The adoption of the Google Cloud Platform resulted in substantial cost savings for the business. Additionally, the implemented solution markedly enhanced data quality, availability, and security. Through the optimization of processing millions of daily events at a petabyte scale, we achieved significant cost reductions. In comparison to the previous Hadoop solution, the utilization of BigQuery enabled the processing of similar queries with a minimum of 15 times greater speed, thereby delivering more precise outcomes.

Contact us to see how we can help you.
We’ll get back to you within 4 hours on working days (Mon – Fri, 9am – 5pm).