Package version:

Interface SynapseSparkJobDefinitionActivity

Execute spark job activity.

interface SynapseSparkJobDefinitionActivity {
    arguments?: any[];
    className?: any;
    conf?: any;
    configurationType?: string;
    dependsOn?: ActivityDependency[];
    description?: string;
    driverSize?: any;
    executorSize?: any;
    file?: any;
    files?: any[];
    filesV2?: any[];
    linkedServiceName?: LinkedServiceReference;
    name: string;
    numExecutors?: any;
    onInactiveMarkAs?: string;
    policy?: ActivityPolicy;
    pythonCodeReference?: any[];
    scanFolder?: any;
    sparkConfig?: {
        [propertyName: string]: any;
    };
    sparkJob: SynapseSparkJobReference;
    state?: string;
    targetBigDataPool?: BigDataPoolParametrizationReference;
    targetSparkConfiguration?: SparkConfigurationParametrizationReference;
    type: "SparkJob";
    userProperties?: UserProperty[];
}

Hierarchy (view full)

Properties

arguments? className? conf? configurationType? dependsOn? description? driverSize? executorSize? file? files? filesV2? linkedServiceName? name numExecutors? onInactiveMarkAs? policy? pythonCodeReference? scanFolder? sparkConfig? sparkJob state? targetBigDataPool? targetSparkConfiguration? type userProperties?

Properties

Optionalarguments

arguments?: any[]

User specified arguments to SynapseSparkJobDefinitionActivity.

OptionalclassName

className?: any

The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).

Optionalconf

conf?: any

Spark configuration properties, which will override the 'conf' of the spark job definition you provide.

OptionalconfigurationType

configurationType?: string

The type of the spark config.

OptionaldependsOn

dependsOn?: ActivityDependency[]

Activity depends on condition.

Optionaldescription

description?: string

Activity description.

OptionaldriverSize

driverSize?: any

Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

OptionalexecutorSize

executorSize?: any

Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

Optionalfile

file?: any

The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).

Optionalfiles

files?: any[]

(Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.

OptionalfilesV2

filesV2?: any[]

Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.

OptionallinkedServiceName

linkedServiceName?: LinkedServiceReference

Linked service reference.

name

name: string

Activity name.

OptionalnumExecutors

numExecutors?: any

Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).

OptionalonInactiveMarkAs

onInactiveMarkAs?: string

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

Optionalpolicy

policy?: ActivityPolicy

Activity policy.

OptionalpythonCodeReference

pythonCodeReference?: any[]

Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.

OptionalscanFolder

scanFolder?: any

Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).

OptionalsparkConfig

sparkConfig?: {
    [propertyName: string]: any;
}

Spark configuration property.

sparkJob

sparkJob: SynapseSparkJobReference

Synapse spark job reference.

Optionalstate

state?: string

Activity state. This is an optional property and if not provided, the state will be Active by default.

OptionaltargetBigDataPool

targetBigDataPool?: BigDataPoolParametrizationReference

The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.

OptionaltargetSparkConfiguration

targetSparkConfiguration?: SparkConfigurationParametrizationReference

The spark configuration of the spark job.

type

type

Polymorphic discriminator, which specifies the different types this object can be

OptionaluserProperties

userProperties?: UserProperty[]

Activity user properties.

Settings

Member Visibility