Mastering Snowflake Copy Command: Exploring 'ON ERROR' Option while loading data

The "ON ERROR" option within the Snowflake COPY command allows users to specify how the system should handle errors encountered during the data loading process. This option provides flexibility in managing data quality issues, ensuring smoother and more efficient data ingestion workflows. ----- SQL QUERY https://drive.google.com/file/d/10nT7nHGAa_GXo0tUFnHXviz_IC2bKNCU/view?usp=sharing ---- Python scripts: --------------------------------------------------------------- import random import csv # Define the categories and subcategories categories = ["Electronics", "Books", "Clothing"] electronics_subcategories = ["Laptops", "Headphones", "Smartphones", "Accessories"] books_subcategories = ["Fiction", "Non-Fiction", "Science"] clothing_subcategories = ["T-Shirts", "Jeans", "Hoodies"] # Function to generate random data for a row def generate_row(): order_id = "ORD" + str(random.randint(1, 10000)).zfill(4) amount = random.randint(50, 500) profit = int(amount * random.uniform(0.1, 0.3)) quantity = random.randint(1, 5) category = random.choice(categories) if category == "Electronics": subcategory = random.choice(electronics_subcategories) elif category == "Books": subcategory = random.choice(books_subcategories) else: subcategory = random.choice(clothing_subcategories) return [order_id, amount, profit, quantity, category, subcategory] # Generate 1000 rows of data rows = [] for _ in range(1500): rows.append(generate_row()) # Write the data to a CSV file with open("orders_dataset_1.csv", "w", newline="") as csvfile: writer = csv.writer(csvfile) writer.writerow(["ORDER_ID", "AMOUNT", "PROFIT", "QUANTITY", "CATEGORY", "SUBCATEGORY"]) writer.writerows(rows) print("Dataset created successfully!") -------------- #Snowflake #DataLoading #ETL #SQL #CloudComputing #SnowflakeTutorial