Introduction to the File Management Tool
What is the File Management Tool?
Use the File Management tool to carry out a number of automated actions on single or multiple files — move, copy, rename, and delete — either locally or across the network. Uploads to and downloads from an FTP server are also supported.
File Management Tool Features
- Copy or move any file type to another location.
- Ability to automatically create the specified destination folder for the file at run-time
- Ability to overwrite a file
- Ability to set the file Read Only attribute
- Limit the moving of a file by setting a maximum file size to process
- Use the existing filename as part of the new name or specify a completely new name
- Add prefix characters to the new filename
- Add a time and date stamp prefix to the new filename
- Add suffix characters to the new filename
- Add a date and time stamp suffix to the new filename
- Preview the new filename based on the selections made before saving them
- Restrict the deletion of a file by the date and time it was last modified
- Limit the copying of a file by setting a maximum file size to process
- Include multiple files for processing through the use of wildcards (
*) when selecting the source file(s)
White Paper - File Management
File Management Tool Technical Summary
File Management Tool – Working with Other Tools
The File Management tool can directly interact with the following tools:
Consuming from Other Tools
The File Management tool can consume objects directly outputted by the following tools:
| Icon | Tool Name | Tool Category |
|---|---|---|
 | Import Flat File | Input |
 | Import XML Document | Input |
File Management Tool – Objects Consumed
The following objects are outputted by the above tools which can be directly consumed by the File Management tool:
- ImportedFileDetails — This object contains the filename and file path of the file previously consumed by the above tools
Exposing to Other Tools
The File Management tool does not expose any objects that can be directly consumed by another tool. However, it does expose the following objects which can be used in a task step sequence:
- FileProcessed — This object exposes the following sub-objects:
- InputFileName — The name of the source file — see General tab
- OutputFileName — The name of the output file if the Rename action has been selected in the Main tab
- InputFilePath — The location of the source file
- OutputFilePath — The new location of the source file if a Move, Copy, or Upload action has been selected
*), the object’s values are overwritten each time the step processed the next file. In such cases, use Memory to compare the file previously processed with the file currently being processed.Error Handling for the Upload and Download Options
Errors that may occur are written to the BPA Platform Event Log for the upload and download options. They include:
- Remote server connection errors at runtime (including account permission errors)
- Read-only errors or other file permission errors
- If the folder that a file is moved to at run-time is zipped
- If the combined total of the selected files sizes for upload or download exceed the amount of remaining space available within the destination folder
Connecting to a File Server
If for security reasons you can only access files over an FTP (File Transfer Protocol), FTPS (File Transfer Protocol (SSL)), SFTP (SSH File Transfer Protocol), or HTTP connection, you must add the connection details to the File Management tool first. If no such security is in place, you can skip this step.
You access the File Management global configuration by either:
You open this window from the resources tree — expand System > Tools > and double-click File Management in the items list.
Click Add to add a new connection.
Adding Connection Properties
The New Connection window of the File Management tool is used to create connections to any network directory.
Choosing the Connection Type
Enter a unique name for this connection.
Choose a Connection Type:
- FTP — The standard network protocol used to transfer files from one host to another. This protocol has a low level of security and does not encrypt the files or user credentials during transfer, therefore not suited for use over a public network where confidential information is involved.
- FTPS — File Transfer Protocol (SSL). This protocol is an extension of FTP which offers security when transferring files. FTP data is protected using SSL encryption (TLS 1.2 protocol)It is suited for the handling of confidential information. Two variants are available:
- Explicit — Once a standard FTP connection is made, a secure SSL connection is then made over the top. The encryption information is then requested. This method has generally overtaken implicit FTPS by the majority of FTP servers.
- Implicit — The SSL connection is made immediately before the encryption information is requested.
- SFTP — SSH File Transfer Protocol. This protocol is over a connection that has previously been secured using the Secure Shell (SSH) protocol. Secure Shell (SSH) is a software-based approach to network security where an installation of an SSH client and an SSH server are required. SSH creates a secure connection when communication is established between the client and the remote server. SFTP is an extension of SSH which adds file transfer capability. It uses the connection to transfer files securely between the two locations. Your files and login credentials are encrypted.
- HTTP — This protocol is used to retrieve files from an FTP Server over HTTP. This would be used when it is not possible to use an FTP protocol, maybe because the file server only permits HTTP connectivity. Alternatively, an FTP client may not be required for a file transfer, in this case a download over HTTP is sufficient.
Adding the FTP Server Details
Enter the Server address to connect to. You can use the hostname, IP address, or a valid URL. By default, the following TCP ports are used to establish the connection to the FTP server:
- FTP — port
21 - FTPS Explicit — port
21 - FTPS Implicit — port
990 - SFTP — port
22 - HTTP — port
80
If the TCP port is use already on your network, adjust this number as required.
Passive mode in FTP is where the FTP connection is initiated from the client side rather than the server. Enable Passive Mode if required by your server.
Server Directory is where the files are downloaded from or uploaded to. Download Directory, on the other hand, is a location on the BPA Platform machine where files are downloaded to.
Provide the required credentials — Enable Anonymous Login if allowed by the FTP server, else clear the selection for this box and enter the required Username and Password.
At this point, you can click Test to ensure the connection is successful.
Configuring Advanced FTP Properties
Click Advanced to add advanced configuration, specific to the Connection Type specified previously:
Step Configuration
To add a new File Management step to an existing task, you either:
- Click and drag the File Management icon from the Task Browser to the task Design area.
- From the task’s Design tab, right-click on empty space and select New > General > File Management.
For a detailed description of how to create new tasks, refer to the product help.
About the General Tab
The General tab is used to enter the following details for the step:
- Name — Enter a meaningful name for the stepTIP: If this task instance makes use of two or more File Management steps, ensure the Name used is unique for each individual step.
- Description — If required, enter a description of this step
Downloading Files Over FTP or HTTP
Enable Download file(s) with FTP or HTTP protocols if this File Management task step is to download files from another server:
- Connection — Contains the global connections created previously.
- Override global server directory setting — If required, enable this if this task step downloads files from a directory not specified in the global connection.
- Directory — Enter the new folder path here.
- File Name — Enter the name of the file to manage. Wildcards, (
*), are supported allowing you to manage multiple files from the FTP location.
Sourcing Files from Previous Steps or Other Locations
If not downloading from an FTP or HTTP server, choose your File source details:
- File Name and Path from previous step — Enable this parameter if a document previously imported into the task is required
- Source — Contains the ImportedFileDetails from those steps capable of importing a document
- Enter a file Path and File Name to select an existing document
Note that File Name is case-insensitive here, for both the filename and file extension.
Wildcards, (*and?), are supported for File Name only, allowing you to manage multiple files in the specified location.
About the Main Tab
Use the Main tab to manage the files.
The following file actions are supported:
Moving Files
Specify the new Destination to move the files to. Create destination folder if not found and Overwrite file if it already exists come into affect at task run-time.
If required, use the settings in the File Read Only Attribute pane to adjust the file’s properties. You must specify the Maximum file size (in MB) that the File Management tool can move. Any matching files larger than the specified size are not moved and remain in their source location. This field is mandatory and defaults to 10 MB.
Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates.
Copying Files
Download Whitepaper
Specify the new Destination to copy the files to. Create destination folder if not found and Overwrite file if it already exists come into affect at task run-time.
If required, use the settings in the File Read Only Attribute pane to adjust the file’s properties.
You must specify the Maximum file size (in MB) that the File Management tool can copy. Any matching files larger than the specified size are not copied. This field is mandatory and defaults to 10 MB.
Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates.
Renaming Files
When renaming files, you can choose to:
- Use input file names as part of the output file name
- Provide a new File Name
The file name, whether new or reused, forms part of the new file name. You can then add a string or a Date and Time Stamp to the beginning of the filename (Prefix file name with) or to the end (Suffix file name with), or do both. At the time of writing, the date and time stamp is formatted as YYYYMMDDhhmmss and cannot be changed.
File name created displays the new file name created at task run-time.
If required, use the settings in the File Read Only Attribute pane to adjust the renamed file’s properties.
Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates.
Deleting Files
You can choose to delete source files based on a specified modification time period of:
- Years
- Months
- Weeks
- Days (default)
- Hours
- Minutes
Download Whitepaper
If required, drag-and-drop a recordset column or variable from the Task Browser to create a dynamic value.
Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates
Uploading Files
When uploading files to an FTP or HTTP server, you must choose the Destination for your source files:
- Connection — Select the global connection for this task step to use.
- Overwrite global server directory setting — In all global connections, you must specify a directory on the FTP server for the connection to use. Enable this parameter to force this task step to use a different folder to upload files to.
- Directory — Specify the new folder path
- Create destination folder if not found — This option only comes into affect at task run-time. Note that if the destination folder is not found and not created, the step will error.
- Overwrite file if it already exists — This option only comes into affect at task run-time. Note that if a matching file is found during the upload and this parameter is not enabled, the step will error.
- Delete source file on completion — Enable this parameter to delete the original source file after a successful upload.
If required, use the settings in the File Read Only Attribute pane to adjust the uploaded file’s properties.
You must specify the Maximum file size (in MB) that the File Management tool can upload. Any matching files larger than the specified size are not uploaded. This field is mandatory and defaults to 10 MB. Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates.
Downloading Files
Download Whitepaper
When downloading files to an FTP or HTTP server, you must choose the Destination for your source files:
- Overwrite global download directory setting — In all global connections, you must specify a directory on the FTP server for the connection to use. Enable this parameter to force this task step to use a different folder to upload files to.
- Directory — Specify the new folder path
- Create destination folder if not found — This option only comes into affect at task run-time. Note that if the destination folder is not found and not created, the step will error.
- Overwrite file if it already exists — This option only comes into affect at task run-time. Note that if a matching file is found during the download and this parameter is not enabled, the step will error.
- Delete source file on completion — Enable this parameter to delete the original source file after a successful download.
- Move source file on completion — If required, you can move the source file to another directory on the FTP or HTTP server.
- Directory — Specify the new folder path. Drag-and-drop recordset columns, variables, and step properties from the Task Browser to create a dynamic folder path. You must ensure the user used to connect to the FTP or HTTP server has the correct access permissions for the new folder.
If required, use the settings in the File Read Only Attribute pane to adjust the downloaded file’s properties. You must specify the Maximum file size (in MB) that the File Management tool can download. Any matching files larger than the specified size are not downloaded. This field is mandatory and defaults to 10 MB.
Use Memory to memorise some or all of the step output for use in other task steps. This is particularly useful when the File Management step is called multiple times — use Memory to compare the document previously processed to the new document currently being processed to avoid duplicates.
About the Options Tab
The Options tab allows you to define how errors in this step are handled at task runtime.
If an error occurs, you can decide whether the step should Continue processing, or terminate the step immediately (Abort Step).
If the step is aborted, you can choose to Continue processing onto the next step in the task, or terminate the whole task immediately (Abort Task).
If file not found when file specification does NOT contain wildcard(s) you can choose to continue processing onto the next task step (Treat as Success) or fail the File Management step (Error).
If no files found when file specification contains wildcard(s) , you can choose to continue processing onto the next task step (Treat as Success) or fail the File Management step (Error).
