awswrangler.emr.build_step

awswrangler.emr.build_step(command: str, name: str = 'my-step', action_on_failure: str = 'CONTINUE', script: bool = False, region: Optional[str] = None, boto3_session: Optional[Session] = None) → Dict[str, Any]

Build the Step structure (dictionary).

Parameters

  • command (str) – e.g. ‘echo “Hello!”’ e.g. for script ‘s3://…/script.sh arg1 arg2’

  • name (str, optional) – Step name.

  • action_on_failure (str) – ‘TERMINATE_JOB_FLOW’, ‘TERMINATE_CLUSTER’, ‘CANCEL_AND_WAIT’, ‘CONTINUE’

  • script (bool) – False for raw command or True for script runner. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-commandrunner.html

  • region (str, optional) – Region name to not get it from boto3.Session. (e.g. us-east-1)

  • boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 session will be used if boto3_session receive None.

Returns

Step structure.

Return type

Dict[str, Any]

Examples

>>> import awswrangler as wr
>>> steps = []
>>> for cmd in ['echo "Hello"', "ls -la"]:
...     steps.append(wr.emr.build_step(name=cmd, command=cmd))
>>> wr.emr.submit_steps(cluster_id="cluster-id", steps=steps)