Back to Browse

Secure, Event-Driven Pipelines With Snowpark Container Services

1.8K views
Jul 18, 2024
22:43

The state of deploying data pipelines has changed considerably for data engineers. Today, many deploy data pipelines using a combination of tools like Airflow and dbt. However, deploying data pipelines outside of Snowflake has led to infrastructure overhead, which has increased scrutiny as moving data from one hop to another is prone to security risks. So how do we take existing data pipelines and deploy them within Snowflake in safe, governed manner? With Snowflake Container Services, it's now possible to do that. In this session, George Yates, Senior Sales Engineer at Astronomer, and Prashant Yadav, Principal Data Platform Architect, Field CTO Office at Snowflake, dive into the architecture considerations of using Snowpark Container Services to deploy your pipelines as event-driven data pipelines. They will also walk through how to leverage current, common pipelines such as Airflow and dbt and deploy them easily as event-driven data pipelines on Snowpark Container Services, which allows the pipeline to be abstracted and consumable. Subscribe for more! http://www.snowflake.com/YTsubscribe/ Explore sample code, download tools, and connect with peers: https://developers.snowflake.com/

Download

0 formats

No download links available.

Secure, Event-Driven Pipelines With Snowpark Container Services | NatokHD