RPG Consuming Web Services with HTTPAPI and SoapUI

Article ID: 56532

Posted April 10th, 2008

By:
Scott Klement
Folks interested in web services will be interested in SoapUI, an open-source (free) tool for
testing and debugging web services. I've found it an invaluable tool when consuming web
services from RPG.

In this article, I review the basics of what a web service is and give you some insight into
how you can test a web service with SoapUI, and after you've tested the web service, I
show you how to use another open-source tool, HTTPAPI, to consume the web service from
an RPG program.

What in Tarnation Is a Web Service?

Think of a web service as a program call, or perhaps a subprocedure call, made over the
web. You call a routine that resides on another computer, using the HTTP protocol -- the
same protocol that your web browser uses to surf the Internet.

Typically, a web service is an API. In other words, it's a routine intended to be called by
another program (as opposed to a user) and performs some sort of service for that
program. It takes all its input as parameters and returns all its output in parameters as
well.

In ILE, we're always being told to write small, reusable routines, so we never have to
repeat the same code in two places. Imagine extending this one step further: Not only is
the code reusable from the same machine, but it can be made reusable from anywhere in
the world, because it's called over the web! That's a powerful idea.

In fact, many software vendors are integrating web services into their products. So if you
purchase their product for (for example) inventory management, you might be able to
integrate it with other applications on other systems within your company's internal
network. The fact that it uses HTTP means you can access it from just about anywhere --
realtime, over the Internet if desired. Of course, you can protect it with things such as
firewalls, SSL, or a VPN if you want to lock down access.

WSDL, SOAP, and XML

Right now, there are two basic types of web services: SOAP and REST. At this point in
time, SOAP is the more standardized and widely used service, and so it's the one I discuss
in this article.

The acronym SOAP stands for Simple Object Access Protocol, though people usually just
refer to it as SOAP, and it's pronounced like "soap," the stuff you use when washing things.

SOAP is a standard format of an XML document that's used to pass parameters to or from
a web service. Basically, when you call a web service, you create a SOAP document
containing all the parameter values that you want to pass into the routine you're calling.
The routine extracts the parameters from the SOAP message, does its processing, and
spits out another SOAP message containing the output parameters. This processing is
usually done while you wait, and in my experience it's very fast.
 How do you know what routines you can call in a given web service? How do you know
what parameters are required? How do you know what the SOAP message should look
like? All this stuff is specified in another XML document. This document is in a standardized
XML format called Web Services Description Language (WSDL), which is always referred to
as WSDL and is typically pronounced like "wiz-dull."

The WSDL document explains everything about a web service, including what Internet
address you need to contact it, what network protocol to use (which, in my experience, is
always HTTP), which character encoding, what the parameters are, and so forth. It tells
you everything you need to know to generate a SOAP message containing the parameters
and send it to the web service.

Here are a few web services terms that you might come across:

o SOAP -- Simple Object Access Protocol; a type of XML document used to encode
parameters to be sent to or from a given operation
o WSDL -- Web Services Description Language; an XML document that describes in full
technical detail, how to use a web service
o UDDI -- a directory of available web services that you can search to find one that fits
your needs; was part of the original web services concept but has never really caught on
o Port -- in ILE terms, a port is analogous to a service program and is a collection of
closely related routines (or "operations") that you can call
o Service -- a collection of related ports; typically there is a separate port for each
network protocol (or "binding") used, but they all fall into the same "service"
o Binding -- specifies the network protocol used to connect to a web service; for REST
web services, it's usually HTTP GET or HTTP POST; for SOAP, it's the SOAP protocol over
HTTP
o Operation -- Each routine that you can call is called an "operation" and is analogous to
a subprocedure in the ILE environment -- or perhaps you prefer to think of each
operation as a routine you can call
o Consume -- call a routine in a web service; a program that calls a web service is called
a "consumer"
o Provide -- provide a web service that can be called; the program that runs on an HTTP
server that a consumer can call is referred to as a "web service provider"
If the terminology seems overwhelming, don't worry about it for the moment. Sometimes I
think learning the terminology is harder than learning the actual technical details.

Basically, to call a web service, you need to find the web service that you're looking for,
then look up its WSDL and use that to determine which operation you want to call and
what the parameters are. I won't describe how to read the WSDL in detail, however,
because nobody does that by hand! Many languages (e.g., Java, .NET, and PHP) have tools
available for them that convert the WSDL to SOAP for you, so you don't need to
understand the WSDL. There's nothing like this built in to RPG, but I'll show you how the
SoapUI tool can make it very easy to consume a web service from RPG.

As an aside . . . there's an open-source tool named WSDL2RPG in the works to

automatically generate RPG code for a given WSDL document -- but that's a topic for
another article.

Testing a Web Service with SoapUI

SoapUI is an open-source (free) GUI tool for testing web services. It's written in Java, and
therefore should run on any platform, though so far, I've run it only on Windows. Currently
two versions are available -- the free one that I'm presenting in this article, and the Pro
version, which isn't free but offers more functionality and commercial support. You can
 download both versions of SoapUI from the following website:
http://www.soapui.org

Assuming that you want to run SoapUI on Windows (like I do), look for the download titled
soapUI-x.x.x-installer.exe. Download it and run the downloaded program to install it on
your system. Then double-click the SoapUI icon to start the program.

To demonstrate SoapUI, I need a web service to test. For the sake of example, I went to
http://www.webservicex.net, a website that offers a directory of easy-to-use web services.
On that site, I noticed a web service called GeoIPService, which tells you the country an IP
address is located in. A nice, simple web service to start with! So I clicked on that
GeoIPService and found that the URL to the WSDL for the service is:
http://www.webservicex.net/geoipservice.asmx?wsdl

To use this WSDL document in SoapUI, I open up the SoapUI program and click File|New
WSDL Project. SoapUI responds by bringing up the following dialog box:

o Project Name -- can be anything you like. This is what this web service will appear as
on the screen, so pick a name that makes sense for your project.
o Initial WSDL -- the WSDL file for the web service that you want to test. It can either be
a path name to a file on your local hard drive or a web URL. In my case, I copied and
pasted the URL of the GeoIPService's WSDL, above.
After SoapUI downloads and interprets the WSDL file, I have a project called Test GeoIP
listed in the navigator pane on the left-hand side of my SoapUI window. To try out the web
service, I expand the GeoIPServiceSoap item, and then the GetGeoIP item. It should now
show an item named Request 1, which I double-click. If all goes well, the window should
now look like this:
 (Click here to see the full-sized image.)

On the left, you'll notice an XML document. This is the SOAP message that will contain the
input parameters that should be sent to the web service. You can click the window
containing this SOAP message and change the XML document, if you like. In this example,
the web service takes only one parameter -- the IP address. SoapUI has placed a question
mark where the IP address should go, so I edit the SOAP message, erase the question
mark, and type in an IP address. If all is well, the web service should tell me which country
the IP address is located in.

After you've inserted the IP address into the SOAP message, click the little green triangle
in the toolbar just above the SOAP message. (It looks something like the Play button on a
CD player.) This action tells SoapUI to send the XML document to the web service and get
back a response. It will display the response SOAP message in the same window. If all is
well, the screen will look like this:

(Click here to see the full-sized image.)

As you can see, in this example the IP address that I typed happens to be located inside
the U.S. Pretty cool, eh? Note that there are little arrows (they look more like less-than or
greater-than symbols) between the SOAP message with the input parameters, and the
output message that I got from the web service. I can click those arrows to switch back
and forth between viewing the input and output messages.

Consuming the Web Service from RPG with HTTPAPI

Now, I'd like to consume this GeoIPService web service from an RPG program. To do that,
I need to generate the SOAP message to send to the server, then send it somehow, get
back the response message, and parse the XML.

Back in 2001, I started an open-source project to let RPG programs interact with HTTP
servers. This tool is called HTTPAPI -- and it's a free tool that anyone can use. It's basically
an RPG service program that you can install on your computer and call from your RPG
program. It handles the network communications of sending data to a web service.
Essentially, HTTPAPI lets your RPG program take the same sort of role as a web browser. It
acts as a client to an HTTP server. (Note: At this time, HTTPAPI does not have tools for
providing web services, only for consuming them.)

HTTPAPI also comes integrated with the Expat utility to help with XML parsing. This XML
support requires V5R1 or later. (Without the XML support, HTTPAPI can be used all the way
back to V4R2.)

You can download HTTPAPI from my website at the following link:

http://www.scottklement.com/httpapi/

The "reference manual" for HTTPAPI is provided in the form of comments included in the
HTTPAPI_H source member of the QRPGLESRC file in the HTTPAPI download. This source
member contains comments explaining what all the routines in HTTPAPI do and what their
parameters are. HTTPAPI also comes with 21 different sample programs named EXAMPLE1,
EXAMPLE2, and so on, up to EXAMPLE21. These demonstrate different ways to use the
HTTPAPI tool.

To use HTTPAPI with the GeoIPService web service, I start by going back to SoapUI and
reformatting the SOAP request (the input parameters) so that each line of the XML
document is no longer than 60 characters. This reformatting is just a matter of breaking
up long lines by moving the cursor to the right spot and pressing Enter to break up the
line. Then, I copy and paste it into an RPG source member and add some plus symbols to
make it into a valid EVAL statement as shown below:

h d f tac tg rp ( *no ) bndd i r ( 'HTTPAP I ' )

/ copy HTTPAP I _H

d GEOIP PR ExtPgm( 'GEO IP ' )

d I PADDR 15a cons t

 d GEOIP PI

d I PADDR 15a cons t

d SOAP s 1024a vary ing

/ f ree

SOAP = '<soapenv:Enve lope +

xmlns : soapenv="h t tp : / / s chemas .xmlsoap .o rg / soap /enve lope / " +

xmlns :web="h t tp : / /www.webserv i cex .ne t "> +

<soapenv:Header /> +

<soapenv:Body> +

<web:GetGeo IP> +

<! - - Opt iona l : - - > +

<web: I PAddress> ' + I PADDR + '< /web : I PAddress> +

</web :GetGeo IP> +

</soapenv:Body> +

</soapenv:Enve lope> ' ;

I hope you can see that this is the exact same XML document as the one I had in the
SoapUI client, except that I've turned it into one big EVAL statement in an RPG program. It
creates the XML document containing the needed input parameter in an RPG variable, and
I've concatenated the IPADDR parameter into the middle, so that the IP address can be
passed as a parameter to my program. (The PR/PI in the D-specs takes the place of the
more traditional *ENTRY PLIST. It merely provides an input parameter.)

Now that I have my SOAP message, I can use HTTPAPI to send it to the web service and
get a response. To do that, I write the following code:

d VARY INGDATAOFFSET. . .

d c cons t (2 )
 d Count ryName s 52a vary ing

d rc s 10 i 0

ht tp_se tCCS IDs (1208 : 0) ;

ht tp_debug(*ON) ;

r c = ht tp_pos t _xml ( ' h t tp : / /www.webserv i cex .ne t /geo ipse rv i ce .asmx '

: %addr (SOAP) + VARY INGDATAOFFSET

: %len (SOAP)

: *NULL

: %paddr (MapXmlData )

: %addr (Coun t ryName)

: HTTP_T IMEOUT

: HTTP_USERAGENT

: ' t ex t /xm l ; charse t=UTF - 8 '

: ' "h t tp : / /www.webserv i cex .ne t /Ge tGeo IP" ' ) ;

i f ( r c<>1) ;

ht tp_c rash ( ) ;

end i f ;

dsp ly Count ryName;

* in l r = *on ;

The http_setCCSIDs() routine tells HTTPAPI to translate my XML document from EBCDIC to
UTF-8 unicode, because this is what the web service is expecting. 1208 is the CCSID for
UTF-8 unicode. 0 is a special value that means "use my jobs default CCSID," which will be
EBCDIC in my environment.

The http_debug() routine tells HTTPAPI to produce a log containing debugging information,
which can be invaluable in troubleshooting HTTPAPI. The debug file will be placed in the
IFS under the name /tmp/httpapi_debug.txt.

The http_post_xml() routine is part of HTTPAPI, and will send the data from the SOAP
variable to the web service. The web address for the web service is passed in the first
parameter to http_post_xml(). How did I know what address to put here? The SoapUI
program listed this address in its window, just above the XML for the SOAP messages.
Look back at the picture of the SoapUI program (above), and you'll see the same URL at
the top of the request window.

The other web address (in the last parameter to http_post_xml) is what's known as the
"SOAP action." To find out what the SOAP action should be, go back to the SoapUI screen
where it shows the input parameters to be passed to the web service. From that window,
click the Raw tab, and you should see the SoapAction listed on the screen.

http_post_xml() will automatically parse the XML that it receives from the web service.
But, because HTTPAPI doesn't really understand SOAP, what should it do with the
response? That's up to you.

As HTTPAPI receives the XML response, it calls a subprocedure -- one that you write -- for
every XML tag that it receives. Note that I've specified %paddr(MapXmlData) as one of the
parameters to HTTPAPI. This tells HTTPAPI that I want it to call the MapXmlData
subprocedure in my program. It will call this same subprocedure many times as it receives
the XML data. It'll call it once for every individual XML tag received. This MapXmlData
routine will get the XML tag name as a parameter, as well as the value of that tag -- which
is to say, the value of the character data inside the tag.

You'll also note that I specified %addr(CountryName) as a parameter to http_post_xml().

This tells HTTPAPI to pass the CountryName variable as the first parameter to the
MapXmlData subprocedure. With that in mind, here's how I wrote the subprocedure:

P MapXmlData B

D MapXmlData PI

D count ry 52a vary ing

D depth 10 I 0 va lue

D name 1024A vary ing cons t

D path 24576A vary ing cons t

D va lue 65535A vary ing cons t

D at t r s * d im(32767)

D cons t opt i ons ( *var s i ze )

 / f ree

i f (name = ' Coun t ryName' ) ;

count ry = va lue ;

end i f ;

/ end - f ree

P E

If you look back at the response message in the SoapUI document, you'll see that the
country name is returned in an XML tag named CountryName. All I've done in the
MapXmlData subprocedure is search for that CountryName XML tag. When I find it, I pass
the value of the tag back to the mainline of my program by setting the 'country' parameter
to the value of the tag.

Because this procedure is going to be called repeatedly, in a loop, for every single XML
element in the response document, I'm always careful to write my code so that it does
nothing when it's passed an XML tag that I'm not interested in. If you look at the code for
the subprocedure, you'll see that if it's called with any other tag name besides
CountryName, it does nothing -- it simply ends without doing any processing.

This MapXmlData() subprocedure is called while http_post_xml() is receiving the XML data
over the network. So when http_post_xml() has ended, this subprocedure has already
done its work and already set the CountryName variable to the value of the CountryName
XML tag. That means that after the call to http_post_xml(), I can go ahead and display the
country name. To keep the example simple, all I did was display it on the screen with the
DSPLY opcode.

Here are some examples of what I see when I run my GEOIP RPG program:

> ca l l geo ip parm( '66 .111 .97 .6 ' )

DSPLY UNITED STATES

> ca l l geo ip parm( '62 .243 .123 .106 ' )

DSPLY DENMARK

Wrap Up

So that's my quick tour of consuming a web service from ILE RPG. I think it's pretty neat
and a lot of fun to do. You can call just about any web service in the world by following
these same steps but using a different WSDL file.

You can download the sample RPG code used in this article at the following link:
http://www.pentontech.com/IBMContent/Documents/article/56532_547_GeoIpService.zip