An Iterator takes an entity set and runs the nested processors using each row of the data.
Iteration can be done serially (Thread Count=1) or in parallel (Thread Count >1).
Iterator must have 1 processor nest under it. For multiple processors, you must add a compound processor at the root of the iterator.
Iterator has the following parameters:
Context Variables – Context Variables allow you to save specific fields values to be used later in a Processor under the Iterator. When a row from an entity set is being processed through several processors inside an iterator, only the first processor receives that row and you might need access to data later in the Iterator.
Parallel Thread Count– Iterators have a parallel thread count setting to have Iterator threads run in parallel. Agent setting stepExecutor.parallelExecution.threadPoolSize (default 20) controls the maximum number of threads allowed. When you have nest iterators, the number of threads is multiplied together.
When running with more than one thread, ensure that the process/data model can be split logically.
Example for Parallel Thread
Reading Projects and sending the result to an Iterator to work on WBS operations for that project.Since each project is independent, it can be used in parallel.
Example for Single Thread
Reading hierarchy data 1 level at a time from 1 system and creating it in another. Running in parallel could have children being created before parents.
Outer Iterator (20 Threads)
Middle Iterator (1 Thread)
Inner Iterator (20 Threads)
This Job requires 20x1x20=400 Threads.
Enter the following parameters in the Basic section,
Enter the name and Parallel Thread Count.
Click the Context Variables field to open the variable selector.
Select the fields which you want to set as variables and name the fields.
These fields can be used later in Processors nested under the iterator using a field assignment with the syntax $__CTX(value)
For example, $__CTX(ProjectObjectId)
You can add other text to the assignment if the use case requires it. For example, ProjectObjectId= $__CTX(ProjectObjectId)’