From Clicks to Code: Managing Starburst Data Products as Code with YAML

This video demonstrates the "data products as code" feature, which is available in public preview. This functionality allows users to manage data products using both a UI and a CLI, facilitating version control through CI/CD integrations such as GitHub. Data Products as Code is available as Public Preview in SEP in the May LTS. Key features and workflows highlighted include: YAML Representation: Data product configurations, including metadata and column descriptions, are represented as YAML files. These files can be stored in a GitHub repository, enabling users to manage changes directly through their preferred code editor. CLI Capabilities: The CLI includes commands for exporting, importing, and overwriting data product YAML files. Users can also perform linting to validate configurations before importing them into Starburst Enterprise (SEP). Workflow Automation: The speaker demonstrates updating a data product by modifying the YAML file, importing the changes, and observing the updates—such as a new owner and an additional dataset—reflected in SEP. Following this, the changes are committed to the GitHub repository and published using the CLI. Natural Language Interaction with Agents: The video showcases the ability to use agents to perform tasks via natural language, such as cloning existing data products and applying modifications like changing owner names or descriptions. Streamlined Management: After making changes via natural language, the agent can lint the configuration, display differences, and handle the import, publication, and synchronization of these changes to the GitHub repository.