Dataflows - Performance and Latency Improvements

We are continuing to expand the significant performance gains delivered by Dataflow Gen2 with a new set of targeted performance and latency improvements across the end-to-end experience. These enhancements focus on the most critical execution paths and user interactions, further differentiating Dataflow Gen2 from previous dataflow generations and raising the bar for low-code data transformations in Microsoft Fabric.This round of improvements addresses performance holistically--spanning data movement, authoring, execution, monitoring, and downstream consumption--to deliver a faster, more responsive, and more predictable experience at scale.Key enhancements include:* Optimized data movement from Data Warehouse compute to Lakehouse destinations, reducing end-to-end refresh times when landing transformed data into Fabric Lakehouse* Reduced metadata refresh latency, improving responsiveness when working with schemas and downstream consumers* Lower latency when creating and editing Dataflow Gen2 artifacts, making authoring and iteration faster and more fluid* Improved refresh history performance, enabling quicker inspection of run status, duration, and failures* Reduced Dataflow connector latency, a critical improvement for downstream consumption and a key enabler for customers migrating from Dataflow Gen1* The ability to turn off Vertiparquet compression, giving advanced users greater control over performance and cost trade-offs for specific workloadsTogether, these enhancements further extend the performance advantages of Dataflow Gen2 compared to earlier dataflow implementations, reducing latency across both runtime execution and day-to-day interactions. By streamlining the most common and time-sensitive operations, Dataflow Gen2 continues to deliver faster refreshes, smoother authoring, and more predictable behavior as data volumes and usage scale.At the same time, these improvements materially enhance the customer experience around low-code data transformations in Fabric. Faster feedback loops, more responsive monitoring, and greater control over execution characteristics make it easier for both business users and data engineers to design, operate, and reuse dataflows with confidence--reinforcing Dataflow Gen2 as a high-performance, production-ready transformation layer within Microsoft Fabric.These performance and latency improvements will become generally available and are designed to be ready for production usage. With GA, customers can rely on these enhancements for business-critical workloads, benefiting from improved stability, predictable behavior, and the operational maturity required to run Dataflow Gen2 at scale across development, test, and production environments.