Queue Manager

Purpose

The Queue Manager keeps a list of all files around that have been detected by the currently active engine. Initially, when a new file has been added to this list, it will be marked as Created. Most engine types will after some time detect that the file has been saved, which means that the file is ready for processing. The engine will then ask the queue manager to update that file’s status to Saved. After some time, assuming there have been no more save events, the file will be upgraded to Queued, meaning that it can now be sent through the operations pipeline for processing. Only when there’s a thread available to do the actual processing, will a Job be created and the file status be changed to Running. Afterwards, the file status will be changed to Success or Failure, depending on the outcome of the pipeline.

If the engine detects another Save event for a file when it has status Queued, then it will simply be demoted to Saved, delaying the processing for that file. However, if the the file is already in status Running, Success or Failure when the Save event is recorded, then the file will be requeued for processing.

Options

The behavior of the Queue Manager may be adjusted by clicking the eponymously named button, which will bring up a dialog with the following options:

• Promote files from ‘Created’ to ‘Saved’: some engines either do not support promoting for Created to Saved, or cannot always be relied on to pick up these Saved events reliably. When this happens, files will be stuck in the queue forever with status Created. This can be avoided by using this option, which will enable automatic promotion to Saved after a selectable number of seconds.
• Delay promoting files from ‘Saved’ to ‘Queued’: sometimes files will be updated multiple files before they can be considered ready for processing. To avoid files being promoted to Queued to soon, it may be useful to increase the minimum amount of time a file has to marked as Saved, before it can be promoted to Queued
• Maximum number of threads to use: this value reflects the number of files may be processed simultaneously. It is limited by the number of CPUs available on the system.
• Remove from table after: when active, successfully processed files will be removed from the table after the requested number of minutes. If the queue manager gets notified of a Saved event for a file that has been removed from the table, it will be added back to it, starting the pipeline all over again.