How to configure the Web Service Connector Tool for RSS feeds

Introduction

This article will show how the Web Service Connector Tool can be configured to enable web service calls to retrieve RSS feed data from a web page. This is achieved by creating a new HTTP based web service configuration which will be used to perform a GET operation on a specified RSS feed at task runtime.

Instructions on how to do so are supplied below.

Creating a new HTTP configuration

As an example, the BBC Weather RSS feed will be used to demonstrate the method of creating a new configuration and applying it at task level. This RSS feed provides a 3 day forecast based on the supplied location via a postcode.

  1. Open the Web Service Connector Tool global configuration and click Add to begin creating a new configuration.
  2. Provide a name for the configuration then select the option “I do not have a definition file, but the web service is based on HTTP…..”. Select OK to continue.New ConfigFigure 1. Adding a new web service configuration.
  3. Within the HTTPWeb Service Configuration dialog, select the Web service root tab. Here you enter the Web Service URL which provides the location of the RSS feed.In this example we areusing:http://open.live.bbc.co.uk/weather/feeds/en/Important: Do not use the Interrogate button as this will overwrite any existing operations.2_ Entering URLFigure 2. Supplying the web service URL.
  4. Select the Operations tab then click Add to create a new single message HTTP operation.
    3_ Adding OperationFigure 3. Adding a new HTTP operation.
  5. Provide a name for the operation and ensure GET has been selected as the Operation Type. Select OK to continue which will open the Define HTTPOperation dialog.4_ Operation TypeFigure 4. Specifying the operation as GET.

    The next few steps will involve building the RSS feed URL which will incorporate parameter fields. A parameter when placed in the URL will be exposed at task level enabling data to be mapped to it. A value inserted into the parameter will effectively complete the URL and retrieve the data required.
  6. Under the URL tab tick the box Use the connection URL. This will insert the web service URLinto the lower text field.
    5_ Building URLFigure 5. Building the RSS feed URL.
  7. Click in the Parameter Name column and enter a new input parameter. The parameter should reflect a parameter field present in the RSS feed that can return results dependant on the value supplied.In this example we have added a single parameter called PostCode.
    6_ Creating parameterFigure 6. Adding a parameter for the URL.
  8. Drag the new parameter to the end of the URL then type the remainder to complete the whole URL for the RSS feed.In this example the PostCode parameter is dragged and added to the URL. The text /3dayforecast.rssis also added to complete the URL which will look as follows:http://open.live.bbc.co.uk/weather/feeds/en/{PostCode}/3dayforecast.rss7_ Completed URLFigure 7. Example shows a completed RSS feed URL. The parameter is dragged into the required location of the URL.
  9. Select OK to save the settings and return to the Operations tab.The operation can now be tested to ensure it will return the desired results and to obtain the XML output.
  10. Click Test which will open the Test Web Service Configuration dialog. Within the Parameter Value field enter a value which can be used by the parameter to return a set of results. The operation can then be tested by clicking the Test button which will display the output within the Results pane.In this example a valid postcode is required to return a 3 day forecast for the area. The value of ‘BH170RB’ is entered. After clicking ‘Test’ the results are returned as XML below.
    8_ Testing the operationFigure 8. Testing the operation to obtain the output.
  11. Copy the Results data then click Close.
  12. Edit your operation again and click the Output Data tab. Tick the box Output required and paste the Results data in to the text box. Be sure to delete the top line “Generated output:“. If the top line is present it will cause an error when trying to infer the schema.
    9_ Copy Results to OutputDataFigure 9. Supplying the XML data.
  13. Click the button Infer schema from XML to JSON. The XML data will be used to generate an XSD schema.
    10_ Infer SchemaFigure 10. Generating an XSD schema from the supplied output XML.
  14. Click the button Validate Schema which if successful will display the following prompt:
    11_ Schema is validFigure 11. Validation successful.
  15. This completes creating a new configuration. Once saved your new operation will be listed ready to be selected for use at task level.
    12_Configuration Created(1)Figure 12. Available web service configurations.

Selecting the HTTP operation at task level

With a new HTTP configuration now available it can be selected from within a Web Service Connector step for data to be mapped to the operation parameters.

Continue as follows:

  1. Create a Web Service Connector step.
  2. Under the Web Service tab select the web service required followed by the operation.In this example, the Web Service drop down list is used to select the web service for our BBC Weather RSS Feed. The Operation drop down list is used to select the Get operation required to retrieve the data.
    13_ Selecting the Web ServiceFigure 13. Selecting the web service and operation to execute.
  3. Select the Mapping tab and use the Functions tab on the right-hand side to add a Fixed/Dynamic data function to the Transformation Mappings pane.This function will be used to insert a value into the parameter of the operation.
    14_1 Adding a function(1)Figure 14. Configuring a data function value for mapping to a parameter.
  4. Double-click the function to enter its properties then select the Value tab. Within the Output Text field you can manually enter a value or drag TaskCentre data from the task browser, such as recordset columns or variables, to dynamically generate a value at task runtime. Click OK to close the function properties once complete.In this example we have manually enter the value of BH170RB.
    15_ Function valueFigure 15. Setting the data function value.
  5. The function can now be mapped toa parameter of the web service operation.Within the Tool Input pane on the right expand the folder Parameters. Select the parameter required and drag it on to the function icon to complete mapping the function to the parameter.In this example, the parameter PostCode is highlighted and dragged onto the function icon in the middle. At task runtime the value ‘BH170RB’ will be inserted into the RSS feed URL for the required data to be returned.
    16_ Mapping the functionFigure 16. Mapping the data function to the parameter.
  6. Click OK to save and return to the task plan.

This completes configuring the Web Service Connector step. When executed at task runtime the step will perform a GET operation on the specified RSS feed.

Linking with other steps

Once the data has been retrieved from the RSS feed, the Web Service Connector tool outputs the data as XML. This means the data can be, for example, saved to a location as an XML file using the Save File tool.

17_ Link to SAF tool

Or, converted to a recordset using the Convert XML to Recordset tool. This will enable the RSS feed data to be passed to other steps in the task for processing.

18_ Link to XML Recordset