Data pipeline support for SparkJobDefinition
Data Factory · General availability · Shipped
Description
Now you can execute your Spark code, including JAR files, directly from a pipeline activity. Just point to your Spark code and the pipeline will execute the job on your Spark cluster in Fabric. This new activity enables exciting data workflow patterns that leverages the power of Fabric's Spark engine while including the Data Factory control flow and data flow capabilities in the same pipeline as your Spark Jobs.
Change History
-
2024-05-21
Roadmap Item Added
Workload: Data Factory