Declarative file data ingestion experience in Fabric Materialized Lake Views
Data Engineering · Public preview · Planned
Description
Fabric data engineers today write 80-100+ lines of boilerplate PySpark to ingest CSV and Parquet files into Delta tables - handling file discovery, schema inference, incremental refresh, schema drift, and error recovery manually. This feature introduces a simple Spark SQL DDL surface (CREATE MATERIALIZED LAKE VIEW ... FROM OneLake_Files OPTIONS (...)) that declaratively handles all of this. It includes automatic schema evolution (or strict fixed-schema mode), three refresh modes (append_only, mirror, full), built-in error handling, structured user telemetry on files and lakehouse-level DAG lineage.
Change History
-
2026-06-02
Roadmap Item Added
Workload: Data Engineering