Real-time personalisation to increase sales opportunities

The challenge

The store's website offers users several actions, such as browsing items, adding or removing products from their basket, among others. The system then uses the resulting data to analyze customer preferences and offer tailored product recommendations based on their interests. Despite this, our client encountered a challenge where the website was not promptly displaying personalised recommendations to users. 

It took up to a week for the system to process and reflect the customer’s actions and decisions in the recommendations, resulting in a negative impact on the customer experience and missed sales opportunities. The client reached out to VirtusLab to optimize the process and minimize the time lag.

The solution

The project initially used Spark batch technology to process data. However, to enhance the data processing capabilities and cater to the business’s growing demands, VirtusLab decided to switch to stream processing services and architectures. This shift allowed VL to ingest a continuous data stream to quickly filter, analyze, transform, or enhance data in real time, combining data feeds from various sources.

The following business analysis showed that implementing Apache Flink on Kubernetes would enable our client to provide users with personalized recommendations in real time based on their past interactions during shopping sessions.

By leveraging Flink on Kubernetes, our client’s online store can ensure high availability and scalability while seamlessly recovering from any system failure without losing any data. Furthermore, it enables them to quickly adapt to rapidly changing situations by easily adding or removing processing units based on website traffic.

The results

Flink is a technology that supports the growth and development of the online store and drives sales through personalized real-time recommendations. With its suitability for programmers and analysts, it has become an essential tool for any modern business. Our client has been able to achieve significant measurable benefits:

1. Efficient processing: The Flink system processes around 122 million rows of data per day (i.e. 1500 per second) about user behavior, then filters and combines these events.
2. Continuous learning: 3,250,000 useful values are employed to train our machine learning models. Processing real-time data enables our client to train machine learning models based on the latest user behavior trends, leading to enhanced personalized recommendations.
3. Improved customer retention: Users are engaged and satisfied, which ultimately leads to increased customer retention.
4. Faster recommendations: The tailored recommendations to customers are now provided within just 15 minutes of receiving data about their behavior, as compared to a week previously.
5. Increased sales: in general, this solution translates into greater profits for the company.
6. Cost savings: By running Flink on Kubernetes, we can integrate technologies to optimize processing units based on website traffic. This directly translates to lower cloud costs as the client is billed based on their reserved computing power, which is only high for periods of most intense traffic.