merge_worker

Merging Workers

A common pattern is to have one worker call another worker. To do this, you need to merge the second worker in the first worker.

Merging a worker, however, is different from gems, files, and other code because the worker need to be uploaded separately and recognized as a worker. In other words, the IronWorker system needs to know it's a worker so can cache it and bring it to the dashboard.

To merge a worker, use the merge_worker command. You want to append the name of the worker as part of the invocation. The merge can be relative but also note that an '.rb' is needed in the name of the file.

merge_worker './status_processor.rb', 'StatusProcessor'

Merged Worker Patterns and Examples

Having workers queue up or schedule workers is an excellent pattern for cloud development. With an elastic task queue, it's easy for an application or a worker to queue up 10s, 100s, or 1000s of jobs. A common pattern is a master/slave arrangement where the master queues up and controls the slave workers. This is useful, for example, in processing streaming data. A scheduled worker can be set to run on a set schedule and can queue up a certain number of workers at that time to handle the data processing. Certain times of day might need more workers than others but that decision could be made by the scheduled worker at run time by inspecting the amount of data that has been collected.

You can see an example of merging workers in other workers in Master/Slave Example in the IronWorker Examples repo as well as in the IronWorker Ruby Gem Test directory on Github.

Notes:

• When using a master/slave arrangement, a few users have suggested making the slave worker a subclass of the master worker (which subclasses IronWorker::base). This has the advantage of keeping the codebase DRY.