<< Click to Display Table of Contents >> Navigation:  Nodes > Input > Triggers > Watcher

 

Purpose

 

This is one of the most important nodes. The Watcher node is useful to automate triggering workflows execution. Use this node to start a queued workflow when new files appear in the folder assigned for this purpose.

As a different to the equivalent in other applications, the Mistika Workflows Watcher node is content aware. Every time a new file lands into the watched folder, the watcher understands if that new file is part of an enumerated sequence, or a checksum file, or a movie file, a MegaPack  content (such us P2 folders), a sidecar metadata file,  or if it is just a standalone file.

When the file (or composite content) stops growing (and other files corresponding to the same content stop landing in the watched folder) then and only then the Universal Paths for the new content found are created and sent to the output connector, and the workflow execution is triggered.

This node does not require manual execution to start the operation. Just add it to the Queue panel to activate it, and the workflow will be executed automatically every time a new content appears in the assigned folder (or optionally in any sub-folder of it).

 

This node does not block the Queue panel while waiting for new files, it just waits silently, and the workflow containing is executed every time new files are detected.

 

For the case of network drives, the Watcher node can only monitor volumes that are already mounted, it can not login to cloud services or similar to check for new content. If you need to monitor a remote volume that is not mounted then use the Auto Trigger node instead, which can start a workflow periodically.

 

Note: The Watcher node is continuously active waiting for new files landing in a folder. If you only want to trigger workflows for the existing files in a folder just one time and then finish, use the Folder node instead.

 

Note: When you place the workflow in the queue, in the first run node the Watcher node will process pre-existing files, and from that point it will only react to new files and files that have been changed.

 

You can watch a basic video tutorial using the Watcher node here

 

Outputs

 

To: The list of new files (Universal Paths) detected in the watched folder that have been stable for a period of time, as defined in the Latency property.

 

Standard properties

Object Name

Color

Bypass

File

Name Convention

Lock

 

Folder properties

Include

Exclude

File Name Only

Filter Mode

Alphabetical Order

Include Subfolders

Add Root

 

 

Node specific properties

 

Latency: It is the time in seconds that the Watcher will wait to see if a media content stops growing. One file is considered to be growing if its size grows during this period of time, or if a related file of the same media pack lands under  the watched folders tree (for example, another frame for the same frame sequence, or another file of the same MegaPack content).

 

Once the related content stops growing, the Watcher will send it to its output and the rest of the workflow is executed.

 

Adjust this value according to your needs. A value too high may delay the task completion, but a value too low may provoke the contents to be processed before they are completely copied or rendered by another program, thus generating a workflow error.

 

ForceCheck: This value is used in systems where the watched folder is not local to your system (it could be a network mounted drive, a SAN drive, etc). In those cases, the operating system may  not detect when a new file appears so it will not notify Mistika Workflows about it. For that reason Mistika Workflows needs to actively monitor the folders looking for new files.

 

The value "0" means to disable it. You can disable  this check to reduce the workload of the system and provide a faster reaction as long as the assigned folder is under control of the local operating system (Then the appearance of local files is detected automatically by the operating system, which will then signal the Workflows application).

 

Delete After Processing: If enabled, the new files found in the watched folder will be deleted once the workflow execution is completed. The files are deleted only if the workflow is successfully completed, and not in the case of errors.

 

As this option can be risky, when this option is activated a new "Are you Sure?" property  will appear asking you for a double check confirmation, which needs to be enabled as a safety measure against involuntary mistakes.  

 

It is important to note that only the processed files are deleted. If there are files in the watched folders that are not included, or excluded from the execution (using the Include / Exclude properties), they will not be deleted.

 

Global Timer: If enabled, all the content under the watcher folder will share the same timer, meaning that no content will be processed until there is no modifications of any file for a while, even if the modified file belongs to other content. For example, if you have a first enumerated sequence already in the watched folder, and a second one is being copied to it then none of the two sequences will be processed until the second one is completed (and no more modifications happen during the timer interval).

 

Having the Global Timer activated is recommended for the case of composite megapacks (Panasonic, XDCam, Stereo3D pairs, etc), because this content is typically made of extremely large files that are also related between them, so if one of those files is already copied while a huge companion file is still being copied then the Watcher node could try to process the first file too early, without waiting for its necessary companion. Having the Global Timer activated will prevent this situation.

 

But in the other hand, the Global Timer has the potential problem that if  the watched folder is extremely busy (if there are constant modifications being made to files on it) then the workflow will never be triggered. Deactivate the Global Timer if you want to prevent this situation.

 

cont.prev.execution (Continue Previous Executions) / Number Of Tries: These properties are useful for reacting to software or system crashes. It permits to decide what to do with the files that were being processed at the crash time:

 

When cont.prev.execution is active, on next execution the Watcher node will retry any files that started a workflow in a previous occasion but didn't complete it (typically due to a crash, blackout, or similar situation).

 

The Number Of Tries property permits to establish a limit of retries, so if a particular file is crashing workflows every time then it will be skipped after that number of retries.

 

If cont.prev.execution is disabled then those files will be ignored.

 

These properties are especially useful in combination with making Mistika Workflows to automatically restart after crashes. How to do that depends on each operating system, but the idea is simple:  First you set the task scheduler of your operating system to restart the workflows  application on system restarts (after auto-login) and also if the application crashes, executing something like:

 

workflows -A Your_mwf_file.mwf

 

And then you can use the above properties to decide what to do with the files that were being processed during the previous crash.

 

 

Contextual menu specific options (Reset, Ignore current files):

 

The Watcher node provides additional menu options in the contextual menu (when clicking with right mouse button over the node):

 

Contextual Menu -> Reset: The Watcher node remember which files have already been executed to make sure they are only executed one time. For this purpose, the list of executed files is stored in a private file in the watched folder (named something like .wfWatcherCacheV2_Watcher.dat file ). The Reset function deletes this file, so all the existing files will be processed again on the next Watcher execution. It is typically useful for repetitive testing.

 

Contextual Menu ->  Ignore current files: It will force to ignore all the pre-existing files in the watched folder (files existing before the first Watcher execution).  Logically It only works if it is executed before adding the workflow to the queue panel, otherwise it will be too late and the Watcher will process all the pre-existing files.

Developer reference (API Rest - node/Watcher/property/[Property ID])