Principles of Metadata-First Design
A 2-day intensive workshop for software developers, engineers, and technical leaders who want to experience metadata-first benefits first-hand.
This immersive workshop helps your teams integrate metadata-first thinking and systems design into everyday architecture and code decisions. It focuses on making metadata explicit in your models, APIs, and services so it becomes a first-class part of your architecture rather than an afterthought, making your AI and LLMs even smarter.
Each workshop will help identify your organization’s systems, constraints, and pain points, so the content feels immediately relevant. Across two days, participants work through focused exercises, quizzes, and hands-on labs that reinforce each concept and ensure they can apply the practices immediately.
Course Outline
Across two intensive days, your team moves from core concepts to practical application and a practical, actionable roadmap you can use in your environment.
Day 1: Build the Machine
| Block | What You’ll Do |
|---|---|
| Environment Setup | Provision warehouse, databases, schemas, and roles from a single script |
| AI Toolkit | Learn metadata-as-context prompting patterns for effective AI use throughout |
| Metadata Catalog | Design and populate SOURCE_SYSTEM, SOURCE_TABLE, SOURCE_ATTRIBUTE with dimensionalized reference tables |
| Staging & Load | Create external stages, file formats, and load raw CSV data into Snowflake |
| Metadataize It | Build LOAD_SOURCE_TABLE — a single procedure that provisions and loads ANY source table from catalog metadata, with 6 configurable load strategies |
| Load Strategies | Validate UPSERT, SCD Type 2, and Soft Delete by switching strategies via metadata — zero code changes |
| Silver Layer | Build CURATE_SOURCE_TABLE and CURATE_SOURCE_VIEW — metadata-driven curation that adapts to new sources automatically |
| Operational Debrief | Query PROCEDURE_LOG to monitor execution history, performance, and failures |
| Principles and Conventions | Principle and conventions that make people more productive and less likely to incur technical debt |
Day 1 takeaway: “You learn and use metadata-first principles to build and run an engine that loads, curates, and monitors ANY source system from metadata alone. Adding a new source = INSERT rows, not write code.”
Day 2: Prove It Works & Make It Discoverable
| Block | What You’ll Do |
|---|---|
| Reconnect | Review Day 1 execution logs; discuss CI/CD implications of metadata-first |
| Quality Framework | Establish a test registry where each test is a ROW, not a script — with scope, severity, and actionable fix guidance |
| Quality Measurement | Populate 16 tests across 5 categories; build a metadata-driven test runner that executes all tests from a single CALL |
| Metadata Search | Build a metadata search procedure that finds tables and columns by name across the entire catalog — with SQL injection prevention: Perfect for AI |
| Data Search | Build a data search procedure that finds values across all data cells; compare live scan vs. pre-computed index for performance |
| Orchestration Preview | See how the same catalog powers scheduling, monitoring, and multi-entity builds in the addon modules |
| Takeways | Summary of principles, conventions, and best practices for metadata-first design |
Day 2 takeaway: “The quality framework tests your data. The search procedures expose your data. Both read from the same catalog you built on Day 1.”
You’ll Learn How to:
- Create and build a fully populated metadata catalog (META.CATALOG)
- Create and populate a working bronze layer with automated, strategy-configurable loading
- Build a curated silver layer generated entirely from metadata
- Write quality tests with a self-service runner
- Build and use search and discovery tools over your catalog and data
- Extend this architecture into MDM, Governance, and Streamlit addons in a principled way using best practices
Prerequisites
- Basic SQL proficiency
- Basic data design familiarity (databases, schemas, tables, procedures)
- No prior metadata framework experience needed
This workshop is designed for professional developers, engineers, and technical staff who are comfortable working with code and system design, but you do not need to be an expert in metadata or have prior formal training in this area.
ShillerCo 