Data Engineering|Published: Aug 20, 2021
Table schemas in data pipelines Spark: How to handle large, nested & growing ones
In this post, we describe how we built a pipeline for the type of “incoming data” situation, and how we came up with a good solution in the end.
