Exposing XML attributes or restructuring XML data using web service connector multi-message operations

Problem Definition

The Convert XML to Recordset tool has limitations in the data output from other steps that it can process (TaskCentre 4.5 Service Release 4 Hotfix H). It cannot recognise attribute data from XML inputs but it cannot deal with certain non-hierarchical structures of XML.

Principles of the Solution

The Web Service Connector Tool has a built in data transformation component (DTC). This can take incoming XML and produce a desired output structure via direct or transformed mappings. This DTC also has the advantage of recognising attributes as input.

The principle of this solution is to use the DTC within the Web Service Connector tool to transform incoming non-hierarchical XML, possibly containing attributes, into a “flattened” form with just elements.

Note that there is a section at the end of this document which may prove helpful for defining schemas.

Solution 1 – Create a Transform Only (Service-less) Operation

Description

This solution will create a new Web Service Connector configuration separate to any actual web services. It is often called a service-less configuration since it will not point at any particular web service. The configuration only contains transforming multi-message operations.

This solution is suitable for general transformations of data even when neither source nor destinations are web services.

Steps

    1. Navigate to System -> Tools ->Execute and double click Web Service Connector to open the tool global configuration.
      1
    2. Click Add to create a new web service configuration. Provide a name and select the last option “I do not have a definition file…” followed by OK.
    3. This will present the HTTP Web Service Configuration dialog. Click the Operations tab.
    4. In the lower portion of the screen there is the option to define a Multi-Message Operation. Click the Add button and provide the operation with a suitable name.
    5. In the Operations tab double click the Input Data icon.
      2
    6. This opens a dialog that allows you to define the flat data structure that you want to create.
      3
    7. Repeat the previous two steps but this time using the Output Data icon. Make the output data structure the same as the input data.
    8. Click the Mapping tab, click the folder containing the elements of the input data and drag it across to the same folder of the output data.
      4
    9. A dialog will pop up asking if you want to generate mappings to which you respond Yes. The mappings will be created as follows:
      5
    10. Click OK to save the mappings then click OK again to save the Multi Message Operation. When closing the Configuration dialog you will be reminded that this type of configuration is only suitable for Multi Message Operations. Click Yes.
    11. The Multi Message Operation is now available for use in a task step. The web service and flat operation can be selected in a web service connector tool task step.
      6
      7
    12. Mappings can be created from deep complex structures to a flat structure and attributes can be mapped into simple consumable elements. This flat, attribute free data can be consumed by the Convert XML to Recordset tool.
      8

Example

When requesting data from Youtube playlists the subtitle is at a higher level than the entry title and entry statistical information is in another level. Furthermore the statistical information is in the form of attributes which cannot normally be processed by the XML2RS tool.

To obtain the attribute data for use in other steps the data needs to be transformed to look like a normal XML element. At the same time it is advantageous to “flatten” the structure, removing sub-levels.

So in the Youtube web service a multi-message operation is defined to “flatten” and expose the required information.

The inputdata for this multi-message operation will still define something like this

InputData

     PlayListEntry

          Title

          SubTitle

          ViewCount

          FavoriteCount

Create a similarly structured OutputData entry.

OutputData

     PlayListEntry

          Title

          SubTitle

          ViewCount

          FavoriteCount

On the OutputData Mapping tab make one-to-one mappings between the entries.

In the task itself, you can now use the normal operations but when you wish to get attributes or “flatten” the data from multiple levels into just one or two (or more accurately, to make them into a consistent “parent-child” hierarchy) you can use the new multi-message operation and pass in the expected data. The main thing to note thereafter is that the loops are correct otherwise no data will be produced.

Solution 2 – Create an operation specific MULTI-MESSAGE OPERATION

Description

An alternative to creating a multi-message operation separate to the main functions of the service would be to produce a multi-message operation that includes the actual service call. This requires that all input data for the call are known.

Steps

    1. Follow steps similar to Solution 1 above, but for the Multi Message Operation define the required input data correctly to be able to make a call to the specific operation.
    2. Drag and drop the true web service operation into the multi message operation.
      9
    3. Open the specific web service operation by double clicking it and map the input data from the left to the right hand side appropriate items.
    4. Define the output data to be the flattened structure you require and map the output from the specific service into the flattened, attribute free structure. If the service operation can return many items, you should define a loop to obtain these multiple details as part of the same XML output. The output from this Multi Message Operation will be in the form required for use in Convert XML to Recordset steps.