What the animation shows

id,amt · {"id":7} · POST: Raw records arriving in each source's own format
Bronze: Landed as-is: mixed sizes, still loose
Silver: Duplicates fade out; records with a red corner failed a quality rule and get fixed (✓) or quarantined; PII is masked (🔒 ███-██-6789)
Gold: Modeled into fact and dimension tables that keep every record. Each destination reads its own tables, one at a time: they light in its color, and a small result rides out. Analytics, apps and reports fetch theirs; reverse ETL and file exports are sent theirs
Chat thread: Pops up every so often, bottom right: an incident (alert, retry, recovery), a deploy with passing tests, or a reconcile run replaying quarantined records into bronze
Hover: A source or destination to trace its lineage

Simulated. Each behavior mirrors work I've done on production platforms.

Jonathan Thomas

Principal Data Engineer · Data Platform Architect

digitalsocal@gmail.com
linkedin.com/in/jmtdata
Costa Mesa, CA

Jonathan Thomas

Principal Data Engineer · Data Platform Architect

2006 Operations
2019 Analytics
2022 Engineering
2026 Principal

I design and own enterprise data platforms end to end, from ingestion and modeling to CI/CD, governance, observability, and the dashboards leaders act on. Today that means a lakehouse unifying 40+ systems at Mutual of Omaha Mortgage, built on Microsoft Fabric and now extending onto Databricks.

Focus: Technical lead on an enterprise Databricks migration spanning SQL Server, Snowflake, and Microsoft Fabric
Based: Costa Mesa, CA
Certified: Microsoft Certified: Fabric Analytics Engineer Associate

LinkedIn ↗ Selected work ↓

Systems integrated

40+

Unified into governed medallion layers
Saved per year

$120K

Offshore team replaced in 90 days
SQL Server data

10 TB+

Managed through the move to a lakehouse
Power BI dashboards

40+

Executive, operational, and migration reporting

Hands-on, every week5,126 GitHub contributions in the last year · active 53 of 53 weeks, 219 days

Mostly private work repositories, so GitHub shows counts only. Snapshot Oct 20.

Decisions

The calls behind the platform: what I chose, what it cost, and what it bought. Select one for detail.

A stricter Corporate layer above Gold

Context: The platform serves sales divisions that each run differently for their market, so many of their analytics are specified by the teams themselves and calculate metrics their own way. Finance, performance reviews and loan-servicer feeds can’t work like that.
What I chose: Kept Gold flexible for division-facing analytics and added a Corporate layer above it for data that has to be exact: built only from validated pipelines and certified metric definitions, and used for financial planning and analysis, employee performance monitoring, and integrations with third parties such as loan servicers.
What it cost: Two layers to model and keep in step, and a Corporate metric changes only through validation, never on request.
What it bought: Divisions keep the freedom to measure their business their own way, while finance, performance management and loan-servicer integrations run on one standardized, validated version of the numbers.

Selected work

Platform-scale builds where the engineering changed how the business works with data. Select one for detail.

Migration Validation Tool

Compares migrated tables row by row after normalizing schemas, keys, types, and casing, then has Claude agents write the root-cause analysis and a report for analysts.

React
Databricks Apps
DataComPy
Hashing
Claude

From billing desks to data platforms

One bar per employer; each segment is a role I held there. Select an employer to see what I delivered.

Operations
Analytics
Engineering
Each segment is a role

Role	Start	End	Duration
Mutual of Omaha Mortgage			4 yr 4 mo
Principal Data Engineer	Mar 2026	Present	7 mo
Data Warehouse Manager	Mar 2025	Mar 2026	1 yr
Data Warehouse Administrator	Mar 2024	Mar 2025	1 yr
Marketing Data & Analytics Manager	Jun 2022	Mar 2024	1 yr 9 mo
Ceridian			3 yr 1 mo
Technical Advanced Support Analyst, Sr.	Feb 2022	Jun 2022	4 mo
Senior Application Specialist	Jan 2021	Feb 2022	1 yr 1 mo
Application Specialist	May 2019	Jan 2021	1 yr 8 mo
Informative Research			4 mo
Production Analyst	Jan 2019	May 2019	4 mo
Hallmark Rehabilitation → Genesis HealthCare · Genesis HealthCare acquired Hallmark Rehabilitation in February 2015.			12 yr 8 mo
Reimbursement Analyst	Feb 2015	Jan 2019	3 yr 11 mo
Financial Operations Supervisor	Jan 2014	Feb 2015	1 yr 1 mo
Finance Operations Assistant	Mar 2010	Jan 2014	3 yr 10 mo
Billing / Accounts Receivable	May 2006	Mar 2010	3 yr 10 mo

Mutual of Omaha Mortgage

Jun 2022 – Present · 4 yr 4 mo · 4 roles

Principal Data Engineer Mar 2026 – Present · 7 mo
- Technical lead for the enterprise Databricks migration, directing three data engineers in a 10+ person program.
- Keep one set of architecture, pipeline, and governance standards across four production platforms: legacy SQL Server, Snowflake, Microsoft Fabric, and Databricks.
- Own Unity Catalog governance for the Databricks platform, including catalog and schema design, grants, and data access and security design.
- Built a reconciliation app (React, Databricks Apps, DataComPy, row hashing, Claude agents) that automated migration validation and root-cause analysis.
- Still own the Snowflake platform after the re-org, including its Terraform infrastructure and GitHub Actions deployments.
- Set team standards for PySpark, Git workflow, CI/CD, notebook structure, and data quality.
- Wrote reusable validation code that normalizes schemas, composite keys, data types, strings, and casing so tables compare row by row.
- Databricks
- Snowflake
- Delta Lake
- PySpark
- Spark SQL
- Microsoft Fabric
- SQL Server
- React
- DataComPy
- Claude
Data Warehouse Manager Mar 2025 – Mar 2026 · 1 yr
- Architected a lakehouse integrating 40+ operational systems into Bronze, Silver, Gold, and Corporate data products on Microsoft Fabric, Delta Lake, and PySpark.
- Took a Snowflake platform from concept to production (~250 tables, 10+ TB, the largest dataset in the environment), merging a vendor data share with SQL Server history and feeding Databricks through CDC and query federation and Microsoft Fabric through mirroring.
- Led the Microsoft Fabric data warehouse build with a two-person team while running the legacy reporting platform.
- Introduced source control, environment promotion, and CI/CD for Fabric, SQL Server, and Power BI using Azure DevOps and GitHub.
- Managed 10+ TB of SQL Server data through the move to the lakehouse.
- Set up Snowflake users, SSO, SCIM provisioning, RBAC, streams, tasks, and stored procedures, plus budgets, spend alerts, and a metadata feed into Fabric that tracks spend, job failures, and daily runs.
- Managed Snowflake environments and access as code with Terraform, deployed through GitHub Actions.
- Designed PySpark pipelines for financial, CRM, marketing, telephony, identity, and security systems.
- Designed Direct Lake semantic models that need no scheduled refreshes.
- Wrote Airflow DAGs in Python for internal and vendor file exports, Power BI snapshot reports, and data freshness checks.
- Microsoft Fabric
- Snowflake
- PySpark
- Delta Lake
- SQL Server
- Power BI
- Apache Airflow
- Azure Data Factory
- Azure DevOps
- GitHub
- Azure Log Analytics
Data Warehouse Administrator Mar 2024 – Mar 2025 · 1 yr
- Replaced an offshore team within 90 days, saving about $120K a year.
- Owned Power BI governance, including workspace structure, Entra ID access, lifecycle management, and a tenant-wide data dictionary.
- Added near real-time pipeline monitoring and anomaly alerts using Azure Log Analytics.
- Tuned workloads and SQL Server performance to meet SLAs, and reported platform spend by department.
- Microsoft Fabric
- Snowflake
- PySpark
- Delta Lake
- SQL Server
- Power BI
- Apache Airflow
- Azure Data Factory
- Azure DevOps
- GitHub
- Azure Log Analytics
Marketing Data & Analytics Manager Jun 2022 – Mar 2024 · 1 yr 9 mo
- Led an analytics modernization that improved data process efficiency by roughly 300%.
- Architected ELT and Reverse ETL pipelines with Python, PySpark, Azure Data Factory, SQL Server, and Microsoft Fabric to centralize operational and marketing data and sync it back to business systems.
- Hired and mentored a data team while staying hands-on.
- Delivered 40+ Power BI dashboards and semantic models for executive reporting, operations, and migration validation.
- Built real-time operational reporting on Salesforce and other systems for monitoring and security.
- Set team practices for data quality, modeling, version control, and automation.
- Python
- PySpark
- Azure Data Factory
- SQL Server
- Power BI
- Salesforce
- Azure DevOps
- Reverse ETL

Stack

What I build with, and the systems I've integrated.

Data Platforms: Microsoft Fabric, Databricks, Delta Lake, SQL Server, Azure SQL, Snowflake, ADLS Gen2
Languages: Python, SQL, T-SQL, PySpark, Snowpark, Spark SQL, DAX, Power Query M, C#, TypeScript, PowerShell, Pandas, VBA
Pipelines & Orchestration: Apache Airflow, Azure Data Factory, Fabric Data Pipelines, SSIS, SQL Server Replication, Snowflake Streams & Tasks, Change Data Capture (CDC), Batch & Streaming, Azure Event Hubs
Architecture & Modeling: Lakehouse Architecture, Medallion Architecture, Dimensional Modeling, SCD Type 1 & Type 2, Source-to-Target Mapping, Data Modeling, Data Warehousing, Distributed Processing, ETL / ELT, Data Products, Materialized Lake Views, Corporate Serving Layer
Governance & Compliance: Data Governance, Data Quality Engineering, Metadata Management, Data Cataloging, Entra ID RBAC, Snowflake RBAC, Unity Catalog, SSO / SAML, PII / NPI Handling, FCRA, SOX, HIPAA, Data Dictionary, Data Quality Rules
Cloud & DevOps: Azure, Azure DevOps, GitHub, Git, CI/CD, Environment Promotion, Release Governance, Terraform, Infrastructure as Code, GitHub Actions, YAML, Entra ID
Integration: REST APIs, SOAP APIs, Webhooks, SFTP / File Ingestion, Microsoft Graph API, OAuth2 / Service Principals, Reverse ETL, Salesforce Integration, Secure Data Sharing, Query Federation, Database Mirroring, Power Automate, Zapier, XML, SharePoint
BI & Analytics: Power BI, Direct Lake, Semantic Modeling, Row-Level Security, Executive Reporting, Self-Service BI, Power Apps, SSRS, Databricks Apps, Self-Service Apps
Observability & Reliability: Azure Log Analytics, Pipeline Monitoring, Anomaly Alerting, Performance Tuning, SLA Management, Cost & Capacity Management
AI Engineering: Claude, LLM Workflows, AI Agents, Prompt Engineering, DataComPy
Leadership & Delivery: Hiring, Mentorship, Engineering Standards, Vendor Management, Jira, Confluence
Systems integrated · 40+: Encompass, nCino, SimpleNexus, LoanCare, DMI, CoreLogic, TransUnion, NMLS, Salesforce, Relcu CRM, Velocify, Total Expert, Five9, ADP, Emburse / Certify, Microsoft Defender, Microsoft Entra, Cofense, JAMF, Azure DevOps · sample of 20
Managed ELT & iPaaS (POC and short-term production): Fivetran, Airbyte, Estuary, Talend, TIBCO

Let's build data people trust.

Always glad to talk data platforms, lakehouse modernization, and hard data-quality problems.

LinkedIn ↗

✓ Microsoft Certified: Fabric Analytics Engineer Associate

Jonathan Thomas

Decisions

A stricter Corporate layer above Gold

One Snowflake hub, two consumption paths

Direct Lake instead of scheduled refresh

Selected work

Migration Validation Tool

Job-Fit Scoring Engine

Enterprise Lakehouse Modernization

Analytics Engineering Platform

Cross-Platform Snowflake Data Hub

Platform Observability & Cost

From billing desks to data platforms

Mutual of Omaha Mortgage

Ceridian

Informative Research

Hallmark Rehabilitation → Genesis HealthCare

Stack

Let's build data people trust.