FMCG

Migration to Google Cloud Platform resulted in cost savings for a global leader in CPG industry

Data Engineering

Client

Global FMCG / CPG Company

Date

Services

Data Migration

Technologies

Google Cloud Products: DataProc, BigQuery, Pub/Sub

Challenge

We needed to develop a Data Platform that could effectively monitor interactions across various touchpoints and provide real-time insights into customer behaviours and product usage. The platform was specifically designed to cater to the company’s Machine Learning and Business Intelligence requirements. It had to efficiently gather data from different sources such as web-clicks, mobile app clickstream, CIAM events, and interactions from the Loyalty Management System.

‍

Our approach

To facilitate the migration of parquet files to the Google Cloud Platform from another cloud storage provider, we relied on Google Cloud Storage buckets and the STS service. For the ingestion and transformation of the flat files, we employed pySpark in Dataproc and loaded them into BigQuery. To handle the loading of events from SaaS CDP, we utilized batch Dataflow jobs. Real-time data sources were ingested through PubSub with BigQuery serving as the sink. The orchestration of ETL processes and downstream SQL processing was implemented as well.

‍

The outcome

The Consumer Data Platform within the Google Cloud Platform serves as the ultimate source of accurate information for all business intelligence (BI) and machine learning (ML) models. The platform possesses the capability to efficiently handle the ingestion of hundreds of millions of customer events on a daily basis. By utilizing the toolset provided by the Google Cloud Platform, the marketing technologists were able to successfully move away from the isolated data storage approach, resulting in the ability to gain a comprehensive, real-time understanding of each individual customer.

‍

Business Impact

The implementation of the Data Platform had a significant impact on the company by enabling a shift from the fragmented data approach to a comprehensive understanding of individual customers in almost real-time. The successful integration of the solution was made possible by leveraging the robust scalability, stability, and security features offered by the Google Cloud Platform. This achievement has become a fundamental pillar in establishing customer insights and formulating a worldwide data-driven strategy for the entire organization.

‍