FilePoller

FilePoller polls for files with the specified name or specified file pattern to move it to a specified directory. It provides a set of actions that allow moving files between two directories as well as it scans a specified directory, checks for the existence of any new files or modification of a file for polling.

Configuration and Testing

Microservice configuration can be done in the CPS (Configuration Property Sheet). The microservice supports adding multiple configurations with each configuration having the necessary details for polling.

The figure below illustrates the panel with a single configuration (Polling) added. 

Microservice Configuration

Action

It includes three options to choose from depending upon the type of operation:

• Move: If selected, matched files will be moved/copied from the source to the destination directory .

• Poll: If selected, the directory will be polled for created/updated/deleted files.

• Archive: If selected, matched files will get archived and archived directory gets placed in the destination folder .

Polling Action

It is prompted after choosing the "Poll" option from the Action property. It includes various options to choose files from the directory to poll.

• Create: This option gives the list of files that are created within a given scheduling interval.
• Update: This option gives the list of files that are updated within a given scheduling interval.
• Delete: This option gives the list of files that are deleted within a given scheduling interval.
  
• All: This option gives the list of files that are created, update and deleted within a given scheduling interval.
  
• List: This option gives the list of all the matched files with the File Name specified.

Compute Paths relative to Directory

The path of the directory relative to which the paths of Source Directory and Destination Directory are computed. By default, this points to the FIORANO_HOME directory. If the paths specified for Source and Destination Directories are not absolute, their paths are calculated relative to the directory specified here.

If the path specified for Source Directory and Destination Directory is absolute, the path specified for the Compute Paths relative to Directory property will not be used in the computation of the directory paths.

Source Directory

The directory which holds the file(s) to be polled/move/archive has to be specified in the Source Directory. All the files in this directory whose names match the pattern specified for the File Name property will be processed.

An absolute path or a path relative to the directory specified in the Compute Paths relative to Directory can be provided.

• The path provided here should point to an existing directory.
• The files present in the sub-directories are not considered.
• The directory specified in Compute Paths relative to Directory property will be used in computing the path only if the path specified here is not absolute.
• Since the microservice is configured to run in scheduling mode, all the input configuration will be provided in the CPS. 

Destination Directory

The path of the directory where the target file (with the data received on the input port) is created. An absolute path of the destination directory or a path relative to the directory provided in Compute Paths relative to Directory can be specified here.

While specifying the Destination Directory, ensure that you have permissions to create/modify files present in this directory.

Use Working Directory

This option is shown only if MOVE action was chosen. An absolute path of the destination directory or a path relative to the directory provided in Compute Paths relative to Directory can be specified here.

Working Directory

Shown only when 'Use Working Directory' is enabled. Working directory is used for processing the files, files are copied/moved to working directory temporarily. Files are not copied/moved to working directory.

File name

The name of the file which needs to be polled in the source directory. This property supports patterns such as *.csv and abc *.*, while multiple patterns are not allowed. 

Archive File Name

Name of the archived file. This is included only when the Archive functionality is enabled.

Action if same file exist

If the file with the same name exists in the Destination folder, choose the action to be taken from the below options:

• Overwrite: Overwrites the existing file and hence the old file gets replaced with the new one.
• Rename: Renames the file in the format - "filename + MMddyyHHmmss + counter(optional)" retaining the previously existing files.

If the Rename option is selected, the following properties will be displayed:

• Append TimeStamp
  • Append date-time format
• Append Counter

Append TimeStamp?

Enabling this appends timestamp in the format specified in the Append date-time format property.

• If enabled, the microservice adds a time stamp whose format is provided through the Timestamp format property
• If disabled, no timestamp is added to the file name

Append date-time format

The format in which the date and time are to be appended to the file name.

Append Counter

A counter added to the destination file name. This is used when the file need not be overwritten.

Extensions of the file to be filtered

Extension of the files for which the transfer of files needs to be restricted. This property accepts a comma-separated list of file extensions. Example: *.zip, *.exe, *.dat.

Example

If this property is set to ".exe" and the file to be moved is named as "installer.exe" in the request, FilePoller ignores that request.

Include Sub Directory

Files will be searched inside subdirectories present inside the source directory. If this option is disabled, only the matching files present inside source directory will be moved/copied.

Example

When this property is enabled, Maintain Sub Directory and Sub Directory Regex property (described below) gets displayed 

Sub Directory Regex

By default, the subdirectory regex pattern is [A-Za-z0-9_-]*, which searches in all the subdirectories for the pattern and includes the subdirectories matching the pattern.

Maintain Sub Directory Order

If the matched files are inside the sub directory, then the subdirectory order will be preserved in the target directory on moving the file. On disabling this property, matched files which are inside the subdirectory get moved/copied to the target directory directly. 

Delete After Move/Archive

Enabling this option results in deletion of files from the source directory after moving to the destination directory. If disabled, files get copied to the destination directory.

Persistent

Enable this property to overcome the polling operation data loss that occurs when Peer/HA server crashes. It creates a file in the $FIORANO_HOME/installer/runtimedata/PeerServers/profile1/FPS/run/components/FilePoller/4.0/tif_<EventProcessName>_<VersionNumber>/<FilePollerMicroserviceName> directory which stores the list of files polled under the directory specified in the CPS.

Example

$FIORANO_HOME/installer/runtimedata/PeerServers/profile1/FPS/run/components/FilePoller/4.0/tif_EVENT_PROCESS7_1.0/FilePoller1

• The file created will be named based on the polling name specified under the Add/Modify File Poller Configuration section in the CPS.
• If multiple polling operations are created, different files will be created in the same directory.

If the component is running on HA servers, the file will be created in the following directory:
$FIORANO_HOME/installer/runtimedata/PeerServers/haprofile1/primary/FPS/run/components/FilePoller/tif_<EventProcessName>_<VersionNumber>/<FilePollerMicroserviceName>

If the option is not enabled, the polling operation information will be lost at the time of peer or HA server crashes.

Scheduler Service Configuration

Please refer to the respective section in the Common Configurations page.

Functional Demonstration

Scenario 1

Moving/Copying a specified set of files in the Source directory to the Destination directory.

Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "MOVE" for the property Action.

Sample Input

Sample Output

Scenario 2

Archive a specified set of files in the Source directory to the Destination directory.

Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "ARCHIVE" for the property Action.

Sample Input

Figure 8: CPS configuration when action is set to 'ARCHIVE'.

Sample Output

Figure 9: Output message received

Scenario 3

Polling a set of files in the Source directory based on the Polling Action chosen.

Configure the FilePoller as described in the Configuration and Testing section and use the Display microservice to check the response. In the FilePoller microservice configuration, choose the option "POLL" for the property Action.

Sample Input

Figure 10: CPS configuration when action is set to 'POLL'.

Sample Output

_{Figure 11: Output message received}

Useful Tips

• The microservice runs on the peer server and therefore the file paths and directories mentioned in the CPS should be valid on the machine where the peer server is running. If the microservice fails over to another peer.
  • Ensure that the machine on which the secondary peer server runs have the same path available.