Staging defaults for Dataflow Gen 2 Output destination
Data Factory · Public preview · Shipped
Description
Dataflow Gen2 provides capabilities to ingest data from a wide range of data sources into the Fabric OneLake. Upon staging this data, it can be transformed at high-scale leveraging the High-Scale Dataflows Gen2 engine (based on Fabric Lakehouse/Warehouse SQL compute).The default behavior for Dataflows Gen2 is to stage data in OneLake to enable high-scale data transformations. While this works great for high-scale scenarios, it does not work as well for scenarios involving small amounts of data being ingested given that it introduces an extra hop (staging) for data before it is ultimately loaded into the dataflow output destination.With the planned enhancements, we're fine tuning the default Staging behavior to be disabled, for queries with an output destination that doesn't require staging (namely, Fabric Lakehouse and Azure SQL Database).Staging behavior can be manually configured on a per-query basis via the Query Settings pane or the query contextual menu in the Queries pane.
Change History
-
2026-06-16
Restored to Roadmap -
2026-06-16
Removed from Roadmap -
2024-05-21
Roadmap Item Added
Workload: Data Factory