In this video we cover the best ways of keeping the data in your Snowflake warehouse up to date, and look at the different ways to load data into your warehouse, including Snowpipe, loading from Kafka and using 3rd party data integration tools.
00:00 - Introduction
02:07 - The CloverDX Data Management Platform
04:00 - What we mean by continuous data integration
08:36 - Techniques for implementation of continuous ingestion into Snowflake
09:45 - Using Snowpipe for continuous data ingestion into Snowflake
14:00 - Using Kafka for continuous data ingestion into Snowflake
17:22 - Using CloverDX for continuous data ingestion into Snowflake
19:46 - An ETL data pipeline in CloverDX that prepares and uploads data to Snowflake, including profiling, reshaping, data mapping, applying validations and enrichments
22:06 - Scheduling the pipeline to run automatically
22:59 - Taking data from a Kafka queue, transforming, and loading to Snowflake (an ETLT approach)
25:05 - How to choose the best approach
Read more:
- More on the CloverDX Data Integration Platform: https://www.cloverdx.com/product
- Request a demo: https://www.cloverdx.com/demo
- Follow CloverDX on LinkedIn: https://www.linkedin.com/company/cloverdx/
- More on getting data to a data warehouse with CloverDX: https://www.cloverdx.com/solutions/data-warehousing
