Workflows

Cirrus workflows are the component that puts the “pipe” in “pipeline”. Workflows model a transformation of an input item or set of input items with a process definition into one or more output items via processing from one or more tasks.

Cirrus also provides several mechanisms for modeling workflow dependencies.

The state database tracks the state of items processed at the workflow level.

Workflow topics:

• Writing workflow definitions
  • Simple example
  • Basic step function structure
  • Triggering workflows off one another
  • Workflow best practices
• Batch tasks in workflows
  • pre-batch and post-batch
  • Put it all together with a parallel block
  • Batch retries vs step function retries
• Workflow chaining
  • Filtering output items
• Workflow callbacks