Hybrid cloud and real-time data processing

10 minutes read

Our retail client (one of the global top 3) collects and processes data from their online store on-premises. Our client’s continuous growth in customer data posed a potential threat to business growth since the remaining solution showed processing limitations.

Migrating to a modern, advanced infrastructure was imperative for handling and training complex ML models, enhancing the online customer experience, and optimizing sales strategies.

Download this success story as PDF

Print it out, take it with you to read later, or share it with your peers.Free download

The challenge

Suitable machine learning methods were crucial to managing big data and meeting business requirements. The existing batch learning approach was suitable for regularly received data. However, for our client’s situation, where data was abundant and increased in frequency, scalability and volume became critical factors. 

Therefore, they required a solution with greater elasticity. The hybrid model proved to be the best solution.  This was the time our client reached out to VirtusLab. 

The solution 

VirtusLab provided the client with a manageable setup for data processing, utilizing Azure Machine Learning service, ETL pipelines, and feature engineering. The hybrid cloud and batch offline processing strategy enabled our client to train ML models on data from 5 million customers, with real-time adaptability to customer behavior.

Additionally, the incorporated feedback loop allowed for continuous learning,  supporting the client’s sales optimization strategy by making real-time predictions based on retained and new customer behavior data.

The results

The new hybrid platform provided the retailer with numerous benefits, including:

  • Overcoming limited computing resources and the shortcomings of an unscalable on-prem solution.
  • Processing several hundred terabytes of client data.
  • Creating a large dataset for model training.
  • Accumulating real-time data enabled faster, cost-efficient learning.
  • Advanced website customization in real-time supported online sales.
  • Adapting quickly to changes in customer behavior.
  • Allowing the data team to adopt the approach to other projects with manageable infrastructure from day one.


The tech stack

Languages: Python, Java

Databases: Hive

Eventing platform: kafka, Flink

Infrastructure: Azure, Azure DevOps, Terraform, Terragrunt, CHEF, GitHub Actions

Modeling: TensorfFlow, scikit-learn, Azure Machine Learning

Take the first step to a sustained competitive edge for your business

Let's connect

VirtusLab's work has met the mark several times over, and their latest project is no exception. The team is efficient, hard-working, and trustworthy. Customers can expect a proactive team that drives results.

Stephen Rooke
Stephen RookeDirector of Software Development @ Extreme Reach

VirtusLab's engineers are truly Strapi extensions experts. Their knowledge and expertise in the area of Strapi plugins gave us the opportunity to lift our multi-brand CMS implementation to a different level.

facile logo
Leonardo PoddaEngineering Manager @ Facile.it

VirtusLab has been an incredible partner since the early development of Scala 3, essential to a mature and stable Scala 3 ecosystem.

Martin OderskyHead of Programming Research Group @ EPFL

VirtusLab's strength is its knowledge of the latest trends and technologies for creating UIs and its ability to design complex applications. The VirtusLab team's in-depth knowledge, understanding, and experience of MIS systems have been invaluable to us in developing our product. The team is professional and delivers on time – we greatly appreciated this efficiency when working with them.

Michael GrantDirector of Development @ Cyber Sec Company