Data pipeline support for SparkJobDefinition

Data Factory · General availability · Shipped

Data Factory General availability Shipped
Last Modified: June 16, 2026 Release Date: Q2 2024

Description

Now you can execute your Spark code, including JAR files, directly from a pipeline activity. Just point to your Spark code and the pipeline will execute the job on your Spark cluster in Fabric. This new activity enables exciting data workflow patterns that leverages the power of Fabric's Spark engine while including the Data Factory control flow and data flow capabilities in the same pipeline as your Spark Jobs.

📰 Related Blog Article

Run Spark Job Definitions in Pipelines with Service Principal or Workspace Identity
🔗 blog.fabric.microsoft.com

Change History

  1. 2026-06-16
    Restored to Roadmap
  2. 2026-06-16
    Removed from Roadmap
  3. 2024-05-21
    Roadmap Item Added
← Back to Roadmap 👁️ Watch this Feature View API JSON Workload: Data Factory