External Functions Tutorial | Snowflake

In this demo you'll learn how to expand the capabilities of your Snowflake data pipelines by easily extending them with external services and functionality. To know more about how different companies are using External Function , you can refer this link -- https://www.vebuso.com/2021/04/bringing-extensibility-to-data-pipelines-whats-new-with-snowflake-external-functions/ Prerequisite: Configuring a Snowflake Storage Integration to Access Amazon S3 https://youtu.be/Mje7AEpxsLA Know about API Gateway in-depth-- What is a Proxy? https://youtu.be/Fjexz_nbxiY Concept of Reverse Proxy & API Gateway https://youtu.be/X8VWVF-I7_4 Introduction to API Gateway with in-depth intuition https://youtu.be/8lCX7CaJ4wM Creating a POST Api | AWS API Gateway Passing Data to AWS Lambda https://youtu.be/vgl9q5Ox5LE Working With Query Parameters In API | API Gateway https://youtu.be/JoR7ECrHDtg Create a REST API on AWS | Step by Step Tutorial https://youtu.be/yp4YggJG9nA Usage Plan & API key - Amazon API Gateway https://youtu.be/fxtHYQJt94A Snowflake External Function Creation Steps: ----------------------------------------------------------------------- Step 1: Creating Lambda Function Step 2: Create Rest API Step 3: Snowflake IAM Role Demo 1: -------------- import json def lambda_handler(event, context): # TODO implement print(event) translated=[] for row in event['data']: data=row[1] message="Hello {}".format(data) translated.append([row[0],message]) return { 'statusCode': 200, 'data': translated } Demo 2: --------------- import json def lambda_handler(event, context): # TODO implement print(event) translated=[] for row in event['data']: name=row[1] department=row[2] message="{} works in {} Department".format(name, department) translated.append([row[0],message]) return { 'statusCode': 200, 'data': translated } Snowflake Code: --------------------------- --drop database if required drop database if exists ramu; --Create Database create database if not exists ramu; use ramu; CREATE OR REPLACE api integration hellotesting api_provider = aws_api_gateway api_aws_role_arn = '{Put your AWS Role here}' enabled = true api_allowed_prefixes = ({API Invoke URL}); DESCRIBE integration hellotesting; create or replace external function helloworld (Name varchar,Department varchar) returns variant api_integration = hellotesting as {API Invoke URL}; select Name,DEPARTMENT,helloworld(Name,DEPARTMENT) from "DEMO_DB"."PUBLIC"."EMPLOYEE"; Check this playlist for more AWS Projects in Big Data domain: https://youtube.com/playlist?list=PLjfRmoYoxpNopPjdACgS5XTfdjyBcuGku