Spark autotune

Data Engineering · Public preview · Shipped

Data Engineering Public preview Shipped
Last Modified: March 25, 2024 Release Date: Q1 2024

Description

Autotune uses machine learning to automatically analyze previous runs of your Spark jobs and tunes the configurations to optimize the performance. It configures how your data is partitioned, joined, and read by Spark. This way it will significantly improve the performance. We have seen customer jobs run 2x faster with this capability.

📰 Related Blog Article

Fabric Spark Autotune and Run Series Job Analysis
🔗 blog.fabric.microsoft.com

Change History

  1. 2024-03-25
    Roadmap Item Added
← Back to Roadmap 👁️ Watch this Feature View API JSON Workload: Data Engineering