Introduction

Although the Import Flat File The Import Flat File tool imports a delimited or fixed width file from disk into BPA Platform, extracts the file contents including any file properties, and outputs it as either a recordset or as XML. and Import XML Document The Import XML Document tool imports an XML document to be used and processed by other task steps. To validate the XML documents at runtime, you can either import an XML schema file (XSD), infer from an existing XML document, or manually construct the XSD. tools support wildcards when searching for the import file, it's not obvious how to force the tools to process all found files. This articles describes how to build a task to do just that.

How Does BPA Platform Import at the Moment?

When specifying the file to import, you can use * (all) and ? (specifically here) wildcards:

The above example searches for all text files starting with ORDER. Currently, if more than one file matches the search query, the oldest file is imported and processed. As none of the Input tools use the Memory feature Memory definitions specify the data, generated by a task at run-time, to be memorised. You can save memorised data in an internal Repository, in an ODBC or OLEDB data source, or in an OLEDB data source via a stored procedure., you can not force the Import tool to search and import again unless the task is re-run.

Required Tools

The tools required for this specific purpose include:

►

Import Flat File or Import XML Document — To import the files

►

File Management Use the File Management tool to carry out a number of automated actions on single or multiple files — move, copy, rename, and delete — either locally or across the network. Uploads to and downloads from an FTP server are also supported. — To move the files from "Processing" to "Done"

►

Decision The Decision tool is used to control the flow of a task process, based on the result of one or more expressions. The decision works by testing a sequence of Boolean (true/false) conditions in turn until an expression returns true. When a true condition is returned then that particular branch of the process is followed. — To determine whether there are more files needing importing

►

Call Task The Call Task tool is used to trigger an external task to run. Using variables, BPA Platform data can be passed between the two tasks at run-time, if required. — To restart the task when it has finished processing the current file

►

Schedule The Schedule tool is used to trigger a task to run based on a date and time schedule. Any number of Schedule steps can be created for a task. — To run the task at set times

Other tools may be involved in your task but these fulfil this specific scenario.

Prerequisites

In addition to the folder where the import files are waiting for processing, create a folder to move the imported file to when the task is finished.

Both folders must be accessible by the BPA Platform client running the task.

Extending the Task

We have a basic task where we are importing a file, formatting, and saving it:

The Import Flat File step is using a wildcard to locate the files to be imported. However, this task only imports and processes the first file it finds.

To ensure that all files found are processed, do the following:

Move the Imported File to "Done"

To ensure we do not import the same file each time the task is run, we need to move it out of the monitored folder. We use the File Management tool to do this:

1. Add the File Management tool to the task.
2. Enter a unique Name for this step and consume the file details from the Import step.
   How Do I Do This?

   1. Enable File Name and Path from previous step.
   2. Choose the Import step from the Source drop-drop.

   

3. Go to the Main tab and set this step to move the imported file to the "done" folder created earlier.
   How Do I Do This?

   1. Enable Move in the Select Option pane.
   2. Enter the folder path to the "processing" folder or use Browse to locate it.
   3. Enable Create destination folder if not found — unless you want to generate an error if a folder is not found, it is recommended you enable this by default.
   4. Leave File Read Only Attribute as Do not change.

   5. You must specify the Maximum file size (in MB) that the File Management tool can move. Any matching files larger than the specified size are not moved and remain in their source location. This field is mandatory and defaults to 10 MB.

4. Go to the Options tab and set the required error handling options.
   How Do I Do This?

   The Options tab allows you to define how errors in this step are handled at task runtime.

   If an error occurs, you can decide whether the step should Continue processing, or terminate the step immediately (Abort Step).

   If the step is aborted, you can choose to Continue processing onto the next step in the task, or terminate the whole task immediately (Abort Task).

   If file not found when file specification does NOT contain wildcard(s) (see General tab), you can choose to continue processing onto the next task step (Treat as Success) or fail the File Management step (Error).

   If no files found when file specification contains wildcard(s) (see General tab), you can choose to continue processing onto the next task step (Treat as Success) or fail the File Management step (Error).

5. Click OK to save and close the step.
6. Change the task flow so this step is the last step of the task.
   How Do I Do This?

   To ensure BPA Platform runs the steps in the correct order, the individual task steps must be linked. You do this by clicking a step to display the step controls — click and hold and draw the connector to its successor.If required, you can add a label to the connection. Double-click the connection line and add the label, up to a maximum of 100 characters. Note that this is a pure-text field so variables, formulas, functions, and Task Browser properties cannot be used in this area.

Checking for Additional Matching Files

We use the Decision tool to check whether the Import tool has found any more files to process and if not, terminate the task:

1. Add a Decision step to the task and Name it Any More Files?.
2. Add a new decision branch that continues the task if the Import tool has found matching files.
   How Do I Do This?

   1. Name this branch Yes.
   2. From the To drop-down, select the step that was originally after the Import step; for our example, the Run Crystal Report The Run Crystal Report tool automates the execution of SAP® Crystal Reports®. Recordsets and variables from other task steps can be consumed to dynamically populate the report at runtime. In addition, where the report requires access to a database, security credentials can be passed from in the Run Crystal Report step. step.
   3. Go to the Script tab.
   4. In the Task Browser, change Scope to be Task.
   5. Expand the Import step's node and drag SuccessCount to the Expression text area in the Decision step.
   6. Add >0 after SuccessCount:

      

   7. Save and close this step.
3. Add a Run VBScript The Run VBScript tool is used to create a task step that runs a Visual Basic Script using data produced by other steps in the task. step to the task; no configuration is required for this step other than naming it End Task.
4. Add another decision branch to your Any More Files? step that goes to the new Run VBScript step if the Import tool hasn't found any matching files.
   How Do I Do this?

   1. Name this branch No.
   2. From the To drop-down, select the Run VBScript step.
   3. Go to the Script tab and enable Else.
   4. Save and close this step.

   Tip: We could write a new expression here where the Import step's SuccessCount is =0 but that is an explicit rule. To cover all other values that don't match the Yes branch, use Else (if values match this, do that, otherwise do the other branch).

5. Change the flow of your task so that the Decision step is AFTER the Import step:

   

Restart the Task

The task flow is now such that if there's a matching file to import, it is dealt with; if no matching files, then the task ends. But how do we force the task to restart to look for more files? A Call Task step is needed to "call" the task once it has completed. First, we need to create a global connection to the folder containing the task — even if you have placed your tasks in the top-level Tasks folder, you must still create the connection.

Creating a Dedicated Call Task User

We must create a dedicated user for the Call Task step to use — this prevents clashes with other user accounts that may be logged into the BPA Platform client at the same time:

1. Save and close the task.
2. From the resources tree, right-click on Users and select New User.
3. Enter calltask as the User name.
4. Enable BPA Platform Authentication and enter a password that adheres to your organisation's security policies. Confirm the password.
5. Go to the Server Roles tab and ensure all server roles are cleared.
6. Click OK to save and close the user.

Creating the Global Connection to the Task Folder

Now we can create the global connection to the relevant task folder:

1. From the resources tree, go to System > Tools > Execute > Call Task.
2. Click Add.
3. Enter a Connection Name of Import Tasks.
4. As the tasks are run on the same computer as this BPA Platform client, enter localhost as the Server.
5. Enter the calltask user's details in Username and Password.
6. Go to the Folder tab and select the folder containing these tasks.

Adding the Call Task Step

Finally, we add a Call Task step to the task.

1. Reopen the task.
2. Add a Call Task step to the task design area.
3. Name this step Call Task - Go Back to Beginning.
4. As we're not using any data from other steps in the task, we can go straight to the Connection tab.
5. Choose the connection created previously.
6. Go to the Mapping tab.
7. From the Object drop-down, select the Import Flat File step.
8. From the Operation drop-down, select Queue — this operation calls the task selected for Object. As this task does not depend on any external data, it can start while the current task run completes.
9. Go to the Options tab and set the required error handling options.
   How Do I Do This?

   The Options tab allows you to define how errors in this step are handled at task runtime.

   If an error occurs, you can decide whether the step should Continue processing, or terminate the step immediately (Abort Step).

   If the step is aborted, you can choose to Continue processing onto the next step in the task, or terminate the whole task immediately (Abort Task). By allowing the task to Continue, you can use the error XML received back in a Save File The Save File tool saves output from another task step capable of producing document, as one or more files on the local drive, on-premise file server, external server, or cloud based server. step for investigation purposes, for example.

10. Click OK to save and close the step.
11. Change the task flow so this step is the last step of the task.
    How Do I Do This?

    To ensure BPA Platform runs the steps in the correct order, the individual task steps must be linked. You do this by clicking a step to display the step controls — click and hold and draw the connector to its successor.If required, you can add a label to the connection. Double-click the connection line and add the label, up to a maximum of 100 characters. Note that this is a pure-text field so variables, formulas, functions, and Task Browser properties cannot be used in this area.

Final Adjustment to the Import Step

As your task is now going to check for matching files , we must adjust the Import step so that if a matching file is not found, the task continues onto the Decision step which handles terminating the task gracefully should it need to.

Note that the following procedure applies to both the Import Flat File and Import XML Document tools:

1. Open the Import step and go to the Options tab.
2. Ensure that If no files could be processed or found matching the source file name has Continue enabled.
3. Click OK to save and close the step.
4. Save the task.

Automatically Running the Task

Currently, the task can only be initiated manually (Tasks toolbar > Queue Now ). At the time of writing, BPA Platform cannot start tasks based on new files being added to folders. Instead, we use the Schedule tool to periodically run the task.

1. Add a Schedule step to the task.
2. In the Main tab, configure the required run schedule.
   How Do I Do This?

   Schedule	Description
   Once Only	The task runs once at the specified Run Date/Time then reverts to a "manually run" task. Last Run Time shows when the task was run.
   Minutes	The task is configured to run once every specified block of minutes (1 to 59), starting from the configured Start date/time. To run indefinitely, do not enable End date/time, else enable this parameter and add the date and time this task run ends. Last and Next Run Time shows when the task was last run and when the next run is due, if any.
   Hours	The task is configured to run once every specified block of hours (1 to 23), starting from the configured Start date/time. To run indefinitely, do not enable End date/time, else enable this parameter and add the date and time this task run ends. Last and Next Run Time shows when the task was last run and when the next run is due, if any.
   Days	The task is configured to run either every day or once every specified block of days, starting from the configured Start date/time. To run indefinitely, do not enable End date/time, else enable this parameter and add the date and time this task run ends. Use Run Date/Time to specify the time of day (24-hour format) this task is run. Last and Next Run Time shows when the task was last run and when the next run is due, if any.
   Weeks	The task is configured to run either every week or every specified block of weeks, starting from the configured Start date/time. To run indefinitely, do not enable End date/time, else enable this parameter and add the date and time this task run ends. Select the days of the week this tasks runs. Use Run Date/Time to specify the time of day (24-hour format) this task is run. Last and Next Run Time shows when the task was last run and when the next run is due, if any.
   Months	The task is configured to run either every month, every specified block of months, or during specific months only, starting from the configured Start date/time. To run indefinitely, do not enable End date/time, else enable this parameter and add the date and time this task run ends. Select the specific day of the month or a time of the month for this task to run. Use Run Date/Time to specify the time of day (24-hour format) this task is run. Last and Next Run Time shows when the task was last run and when the next run is due, if any.
   Continuously	The task is run continuously. No start or end time can be specified for a continuous schedule. Note: The continuous option should be used with care due to its resource requirements — if using an internal store with over several hundred tasks, be aware that continuous tasks may take longer to complete. Also, continuous schedules may still queue if there are other schedules waiting to run and no thread available to run them.

3. Click OK to save and close the step.
4. Change the task flow so that this is the first step of the task:

The Schedule step has no impact on the Call Task step's ability to restart the task; any manual task initiation, whether by a Call Task step or by selecting Queue Now in the Task menu, always overrides an existing Schedule step.

For more information about each individual tool used in this article, refer to the Tool-Specific Guides section.