Skip to main content

VirtusLab's ArticlesRSS

Data Engineering|Mar 23, 2023

Is Hadoop still relevant: Is it our future, or does it belong to the past?

As business needs and market trends change, Hadoop and cloud data platforms will evolve together. Will Hadoop remain a data solution companies rely on?

Is_Hadoop_still_relevant__Is_it_our_future,_or_does_it_belong_to_the_past_image-min.jpg
Data Engineering|Aug 20, 2021

Table schemas in data pipelines Spark: How to handle large, nested & growing ones

In this post, we describe how we built a pipeline for the type of “incoming data” situation, and how we came up with a good solution in the end.

Table_schemas_in_data_pipelines_Spark_How_to_handle_large,_nested_&_growing_ones_image-min.jpg