Kapow Mashup Server 6.

Quick Start Guide

Quick Start Guide

Copyright 1999-2007 Kapow Technologies http://www.kapowtech.com All rights reserved.

CONTENTS

iii

Contents
WELCOME TO KAPOW MASHUP SERVER .......................................................... 1 Installation ........................................................................................1 About the Guide .................................................................................1 Contact Information ............................................................................1 Support ..........................................................................................1 Kapow Developer Connection.............................................................1 OVERVIEW OF KAPOW MASHUP SERVER ......................................................... 2 Web Integration .................................................................................2 Enter Kapow Mashup Server.................................................................2 How It Works .....................................................................................3 Robots ...........................................................................................4 Objects ..........................................................................................4 Summary ..........................................................................................4 KAPOW MASHUP SERVER APPLICATIONS ........................................................ 5 Creating Robots..................................................................................5 Running Robots ..................................................................................6 Managing Robots ................................................................................7 DOCUMENTATION OVERVIEW ....................................................................... 8 User Guides .......................................................................................8 Programmer Guides ............................................................................8 Administrator Guides...........................................................................9 Recommended Reading .......................................................................9 Kapow Mashup Server Basics.............................................................9 Data Collection .............................................................................. 10 Clipping ........................................................................................ 10 Integration into Web Applications or Portals ....................................... 10 Integration into other Java Applications............................................. 11 Integration with Microsoft .NET Application ........................................ 11 Java Web Services for BEA WebLogic 8.1........................................... 12 Web Services for Microsoft .NET ....................................................... 12 QUICK START TUTORIAL 1: DATA COLLECTION .............................................. 13 Overview......................................................................................... 13 Notes ........................................................................................... 13 Before You Start ............................................................................ 13 Defining the Tip Object ...................................................................... 13 Step 1 Start up ModelMaker.......................................................... 15 Step 2 Create a New Domain Model ............................................... 16

iv

QUICK START GUIDE Step 3 Create the Tip Object ......................................................... 17 Step 4 Define Attributes for the Tip Object ...................................... 17 Step 5 Validate the Domain Model ................................................. 20 Step 6 Save the Model ................................................................. 20 Creating the Robot............................................................................ 21 Step 7 Start RoboMaker ............................................................... 21 Step 8 Create a new Robot ........................................................... 22 Step 9 Choose Robot Type............................................................ 22 Step 10 Enter the URL to Start From .............................................. 23 Step 11 Choose Output Object ...................................................... 24 Step 12 Loop Through Table Rows ................................................. 26 Step 13 Extract Tip Number .......................................................... 27 Step 14 Extract Tip URL ............................................................... 29 Step 15 Load the Page with the Tip ................................................ 30 Step 16 Extract Tip Title ............................................................... 31 Step 17 Extract Description .......................................................... 32 Step 18 Return Object ................................................................. 33 Step 19 Save the Robot ............................................................... 33 Running the Robot ............................................................................ 33 Step 20 Run in RoboMakers Debugger ........................................... 34 Step 21 Run Using RoboRunner ..................................................... 34 Step 22 Start RoboServer............................................................. 35 Step 23 Start RoboClient .............................................................. 36 Step 24 Run the Robot from RoboClient.......................................... 36 Step 25 Run the Robot from the RoboSuite Java API ........................ 37 Step 26 Run the Robot from the RoboSuite .NET API ........................ 38 Summary ........................................................................................ 39

QUICK START TUTORIAL 2: CLIPPING.......................................................... 40 Overview......................................................................................... 40 Creating the Clipping Robot................................................................ 40 Step 1 Start RoboMaker ............................................................... 40 Step 2 Create a New Robot ........................................................... 41 Step 3 Choose Robot Type............................................................ 42 Step 4 Enter the URL to Start From................................................ 43 Step 5 Configure the URLs to Clip .................................................. 44 Step 6 Add a New Clip Branch ....................................................... 46 Step 7 Enter Name of New Clip Branch ........................................... 47 Step 8 Hide a Button ................................................................... 48 Step 9 Select the Logo ................................................................. 49 Step 10 Select the Search Form .................................................... 50 Step 11 Show the Clip.................................................................. 51 Step 12 Make a Search ................................................................ 52

CONTENTS

Step 13 Ensure the Correct Clip Branch is Used ............................... 53 Step 14 Save the Robot ............................................................... 55 Installing Tomcat.............................................................................. 56 Step 16 Download Apache Tomcat ................................................. 56 Step 17 Install Apache Tomcat ...................................................... 56 Step 18 Install the RoboSuite Tag Library Demo Web Application on Apache Tomcat.............................................................................. 58 Step 19 Test that Tomcat is Running Correctly ................................ 58 Deploying the Clipping Robot.............................................................. 59 Step 20 Start the Java Servlet Clip Wizard ...................................... 59 Step 21 Choose a Name for the Deployment Descriptor File............... 60 Step 22 Finish the Wizard ............................................................. 60 Running the Clipping Robot ................................................................ 61 Step 23 Restart Tomcat................................................................ 61 Step 24 Start RoboServer............................................................. 61 Step 25 Try out Clipping............................................................... 61 Summary ........................................................................................ 62 Next Steps....................................................................................... 62 INDEX.................................................................................................. 63

WELCOME TO KAPOW MASHUP SERVER

Welcome to Kapow Mashup Server

Welcome to the Kapow Mashup Server product suite. In this guide you will get a brief overview of the entire product and its manuals.

Installation
Read the Installation Guide for a description of how to install the product. This is not covered in this overview.

About the Guide

This guide is intended as a quick and not too technical introduction to Kapow Mashup Server. It contains a short introduction to the Kapow Mashup Server product, a description of the applications in the suite, and an overview of the available manuals. We also present a list of recommended reading, necessary to get you started with the product. Finally, the guide contains two simple tutorials, one describing data collection and one describing web clipping. The tutorials will step by step guide you through the two simple cases, introducing you to all the applications of Kapow Mashup Server and how they are interconnected.

Contact Information
Support
Support information is available at this page: http://support.kapowtech.com

Kapow Developer Connection

The Kapow Developer Connection provides access to many resources for developers. It contains the latest downloads, as well as a problem tracking system, and a discussion forum. It is located at this URL: http://kdc.kapowtech.com/

QUICK START GUIDE

Overview of Kapow Mashup Server

Kapow Mashup Server is a platform for Web Integration. In this chapter, we will describe the meaning of that term, and the role of Kapow Mashup Server in bringing about this integration.

Web Integration
Web Integration is the process of leveraging the informational and functional resources of the Web in an application of your choice. There are a number of compelling reasons why such an approach would be interesting. The first and most obvious is the sheer quantity of resources. Second, the Web integration approach allows you to take advantage of existing enterprise information systems (EIS) that provide Web interfaces. The challenges in implementing such a solution from scratch should also be obvious. The resources are there, but they are notoriously heterogeneous, and are often dispersed among several pages. Another complication is the language in which the resources are available Hypertext Markup Language (HTML). The very thing that has contributed to its success (its loose and forgiving syntax) makes it very difficult to impose structure on HTML. Add to this the complexity of client-side JavaScript, user validation and session handling, and the solution becomes non-trivial.

Enter Kapow Mashup Server

Kapow Mashup Server helps transform the resources of the Web into welldefined nuggets of information and functionality. In effect, Kapow Mashup Server transforms a web site into one or more services available to a client application. Most importantly, it does this in a non-intrusive way. Figure 1 illustrates how Kapow Mashup Server accomplishes Web Integration.

Website

Your Application

RoboSuite

Website

Website
Figure 1: Web Integration with Kapow Mashup Server

OVERVIEW OF KAPOW MASHUP SERVER

How It Works
This next figure goes into more detail on the architecture of the Kapow Mashup Server based solution. The central application in executing the integration is called RoboServer. RoboServer receives a request from your application and executes what is called a robot. We will give a more detailed description of what a robot is shortly. For now it is sufficient to know that to integrate with a web site you write a robot that when executed interacts with that web site. This is illustrated in Figure 2 below. Figure 2 also illustrates how different applications interact with RoboServer. A web application may use the RoboSuite JSP Tag Library to call robots from JSP pages. This tag library uses the RoboSuite Java API to contact RoboServer. A Java or .NET application may use the RoboSuite Java API or RoboSuite .NET API to contact RoboServer, and other applications, e.g. a C++ program, may contact RoboServer directly over a socket connection.

JSP Web Application

Java/.NET Application

Other Applications

RoboSuite JSP Tag Library

RoboSuite Java/.NET APIs

RoboServer

Robot

Robot

Robot

Robot

Website

Website

Website

Website

Figure 2: How Kapow Mashup Server Works

QUICK START GUIDE

Robots
Robots are the fundamental concept in Kapow Mashup Server. A robot can be thought of as a small program that performs some sort of interaction or dataextraction from the Web. The programming analogy is not wholly accurate, however. Robots can be written and maintained by non-programmers using a set of intuitive visual builder tools. By navigating the target web pages in a built-in browser, the user defines the actions that the robot will take. In this regard, robots have much in common with macros, and can also be described as macros for the Web.

Objects
Robots are invoked at the request of a client application or as part of a batch job. They can be passed a set of objects as input, and when running they return a set of objects back to the user. The robot developer decides what kind of objects a robot takes as input, and which objects it may return. The objects are modeled to fit real-world objects, like cars, houses, stock quotes, and airplane reservations. After the modeling of objects is done, the robot developer can create the robots that actually perform the tasks, e.g. searching for products (e.g. for a price comparison site), booking airplane tickets, retrieving stock quotes or accessing customer information in a CRM system.

Summary
In this chapter, we explained the nature of Web Integration, and how Kapow Mashup Server fits as the solution for this.

KAPOW MASHUP SERVER APPLICATIONS

Kapow Mashup Server Applications

Kapow Mashup Server is made up of several applications. Figure 3 shows the relationship between the different applications, and how they are used to write, run and manage a collection of robots.
Robot

ModelMaker

creates

uses

RoboMaker

creates

interacts with

Website Website

Object Model

Website Your Application interacts with RoboServer (or RoboRunner) executes Website

RoboManager

works on

generates

monitors

RoboSuite Control Center

Log Database

Log Message

Figure 3: Kapow Mashup Server Architecture Overview

Creating Robots
The most essential role in a solution based on Kapow Mashup Server is that of the robot developer, the person responsible for writing and maintaining robots. The main tools of a robot developer are ModelMaker and RoboMaker: ModelMaker: ModelMaker is an application for modeling the objects used by a robot. These include the objects a robot extracts (and returns to the caller), and the objects it requires as input parameters. RoboMaker: The central tool in Kapow Mashup Server, RoboMaker allows you to create and debug robots. As a user, you should be familiar with these two central applications.

QUICK START GUIDE

Running Robots
Kapow Mashup Server offers several ways to execute robots. They can be run as batch jobs using RoboRunner, or from a client application using RoboServer and an appropriate API or communication protocol. RoboRunner: RoboRunner allows you to run robots from the command line. It provides almost as much functionality as RoboServer, except that it does not allow you to pass objects to the robot. RoboServer: RoboServer allows remote clients to execute robots. It accepts requests and sends back replies to client applications. Control Center: Control Center allows you to remotely monitor RoboServers and their robots. RoboClient: RoboClient is a simple client to RoboServer that allows you to test various requests, and to get a feel for RoboServer. The choice of using RoboRunner or RoboServer depends on what you wish to accomplish. For data collection running at certain intervals, you can use RoboRunner, together with a scheduling tool of your own choice. If you want to do web integration initiated by an end-user, you will need RoboServer. To communicate with RoboServer from your client applications, you may use one of the APIs. Currently, we provide application programming interfaces for the Microsoft .NET platform, the Java platform and the Java Server Pages (JSP) technology. RoboSuite Java API: Provides a complete Java API for accessing every capability of RoboServer. RoboSuite JSP Tag Library: Built on top of the RoboSuite Java API, the tag library provides support for using robots in JSP pages. RoboSuite .NET API: Provides a Microsoft .NET API for accessing every capability of RoboServer. Additionally, for BEA customers, we provide a custom control for BEA Workshop 8.1: RoboSuite Control for BEA WebLogic Workshop: This is an extensible control that allows you to integrate Kapow Mashup Server into the BEA platform through WebLogic Workshop.

KAPOW MASHUP SERVER APPLICATIONS

Managing Robots
Managing a single robot or even a couple of robots is quite easy. Keeping track of hundreds or thousands is a challenge. RoboManager is a tool that helps you deal with this complexity. RoboManager: A tool for organizing and maintaining a large number of robots. RoboManager allows you to monitor how each robot is performing in order to detect robot failures or other problems.

QUICK START GUIDE

Documentation Overview
This section describes each manual in turn and finally provides a list of recommended reading.

User Guides
The users guides describe the applications that make up the suite, one by one. ModelMaker User's Guide: Gives an introduction to ModelMaker, as well as an overview of the object model. RoboMaker User's Guide: A comprehensive guide to writing robots in RoboMaker, and debugging robots in the integrated debugging tool RoboDebugger. RoboManager User's Guide: Describes how to organize and manage a large number of robots using RoboManager. RoboServer User's Guide: Describes RoboServer and RoboClient. The guide also describes the Robot Query Language (RQL) that is used to communicate with RoboServer. Control Center User's Guide: Describes how to monitor RoboServers and their robots using the Control Center. RoboRunner User's Guide: Describes how to use RoboRunner for executing robots. Code Generation Guide: Describes the various wizards for code generation, e.g. Java, C#, and clipping.

Programmer Guides
The programming APIs come with a set of programming guides to complement the users guides. Java Programmer's Guide: Explains how to use the RoboSuite Java API to execute robots on a RoboServer. The target audience for this book is Java programmers of varying skill levels. JSP Tag Library Programmer's Guide: Gives a complete description of the RoboSuite JSP Tag Library that is used to run robots from JSP pages. This book requires basic knowledge of HTML, and some experience with writing JSP pages is recommended. .NET Programmer's Guide: Describes how to use the .NET API for RoboSuite. The target audience for this book is .NET programmers of varying skill levels. BEA Workshop Control Developer's Guide: Provides a description of the RoboSuite Control for BEA Workshop 8.1.

DOCUMENTATION OVERVIEW

In addition to these programming guides, both APIs come with a set of reference documentation, which is installed as part of the installation.

Administrator Guides
Kapow Mashup Server comes with an installation guide that describes how to install and set up the application suite. Installation Guide: Describes how to install Kapow Mashup Server, how to install database connections and how to install certificates. It also details the Settings application. It does not go into detail on how to set up Kapow Mashup Server for particular needs, such as data collection or web integration. RQL Service User's Guide: An advanced guide to configuring RoboServer, and setting up the communication protocols it uses.

Recommended Reading
In this section, we take a look at which parts of the manuals you need to read depending on what tasks you want to perform with Kapow Mashup Server. We present recommended reading for several areas of interest, mainly data collection and different integration approaches. Each category lists the manuals and chapters to read. The rest of the documentation can serve as reference material and as a source of additional information when needed. It is recommended that you start reading the material listed under Kapow Mashup Server Basics below. This will give you a solid foundation for reading further and your first attempts at creating a robot. Depending on the role Kapow Mashup Server will play in your solution, you should then go on to read one of the next three categories of information.

Kapow Mashup Server Basics

This is the required reading for those developing robots. It describes the basic working of RoboMaker and ModelMaker, and is basic knowledge every user of the product should be familiar with. RoboMaker User's Guide: Introduction RoboMaker Basics Getting Started Tutorial Case 1: Essentials Tutorial Case 2: Form Submission ModelMaker User's Guide Introduction ModelMaker Basics

10

QUICK START GUIDE

Tutorial: Creating a Domain Model This should be enough to help you get started.

Data Collection
If you plan to use robots for large-scale data collection, you should familiarize yourself with RoboRunner and RoboManager. RoboRunner User's Guide (The entire guide) RoboManager User's Guide (The entire guide)

Clipping
If you want to use Kapow Mashup Server for clipping web sites and reusing them in another context, you should read the following: RoboMaker User's Guide: How to Clip RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) Code Generation Guide (The relevant clipping portlet generation wizards.)

Integration into Web Applications or Portals

This is what you should read if you wish to use Kapow Mashup Server for integrating functionality from existing Web resources into your own web application: RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) JSP Tag Library Programmer's Guide Introduction Overview After reading these chapters, you should experiment with the Demo Web Application (located as part of the installation in API/robosuite-jsptaglib/examples).

DOCUMENTATION OVERVIEW

11

You may also wish to look through the Clipping documentation (the previous category), to see another approach for providing portlet content.

Integration into other Java Applications

The RoboSuite Java API allows you to use robots from any Java application. Needless to say, you should have some Java programming experience before you attempt to use the supplied API. Before programming directly against the Java API you should try to use the Java Code Generation Wizard and see whether or not it is capable of fulfilling your needs. RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) Code Generation Guide Java Code Generation Wizard Java Programmer's Guide Introduction Core Functionality After this, you should try out the examples that are provided as part of the installation (in the API/robosuite-java-api/examples folder).

Integration with Microsoft .NET Application

The RoboSuite .NET API allows you to use robots from any .NET application. Its main target audience is Microsoft .NET developers. Before programming directly against the .NET API you should try to use the C# Code Generation Wizard and see whether or not it is capable of fulfilling your needs. RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) Code Generation Guide C# Code Generation Wizard RoboSuite .NET Programmer's Guide Introduction Core Functionality After this, you should try out the examples that are provided as part of the installation (in the API/robosuite-dotnet-api/examples folder).

12

QUICK START GUIDE

Java Web Services for BEA WebLogic 8.1

This is what you should read if you wish to use Kapow Mashup Server for creating web services for BEA WebLogic 8.1: RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) Code Generation Guide Introduction BEA Java Web Service Wizard

Web Services for Microsoft .NET

If you wish to use Kapow Mashup Server for creating web services running under Microsoft .NET you should read the following: RoboServer User's Guide Introduction Starting RoboServer Getting a Feel for RoboServer Understanding RQL (optional) Code Generation Guide .NET Web Service Wizard

QUICK START TUTORIAL 1: DATA COLLECTION

13

Quick Start Tutorial 1: Data Collection

This chapter contains a short tour of Kapow Mashup Server. It will guide you through the creation and execution of your first simple robot. It should take about 30 minutes to complete. The goal of the tour is to take you through the core Kapow Mashup Server applications, and show how they function together to help you create and run robots.

Overview
We will be creating a simple robot that goes to the Kapow website and fetches a list of RoboSuite tips of the week. This demonstrates simple data collection. The first thing we do is modeling the type of object that will be returned from the robot. We will call this object Tip, and it will contain attributes that describe the tip. Then we go on to create the robot. It will be a small to medium sized robot that reads the various parts of the tip from the web page. After the robot is finished, we run the robot. First we run it in RoboMakers debugger, then with RoboRunner, then by calling RoboServer from RoboClient, and finally programmatically using the Java API to call RoboServer. The tutorial is divided into a number of small steps, each describing a single action to take. Most steps start out with a short description of what we will do, and two subsections: Action, and Effect.

Notes
During the tour, we make references to the installation folder. By this we mean the directory that Kapow Mashup Server is installed in. On a Windows system, this would be C:\Program Files\Kapow Mashup Server 6.4 if you have chosen the default value at installation. Since the precise location of the folder will vary among users, depending on where you selected to install, we refer to it instead as the installation folder.

Before You Start

Before you can go through this guide, you need to have Kapow Mashup Server installed on your system. If you have not installed Kapow Mashup Server already, now would be a good time. You can find help on how to do this in the Installation Guide.

Defining the Tip Object

The first step in defining the Tip object is to take a first look at the target site and consider what the robot should do.

14

QUICK START GUIDE

In your browser, go to the page http://www.kapowtech.com/tutorial/quickstart. This site is constructed with the sole purpose of acting as an example site for this tutorial and is therefore kept extremely simple. The main page is a table of all the tips, containing the numbers and titles of these. The titles link to a full description of the tips. Take a few minutes to familiarize yourself with the structure of the site. We want the robot to send back information describing the tip of the day. We will choose the name Tip to describe it, and it will have four attributes: title: A short string describing what the tip is about. number: What number tip it is. All tips are numbered incrementally, starting with number 1. url: The location of a page describing the tip. This can be used if we want to present the tip in our own web page and link to the original location of the tip. description: The actual description of the tip, often just a short paragraph of text. About Domain Models In a real application, we would often need more than a single object type to describe the data. We would perhaps need an object type to describe information sent to the robot, as well as several different object types that could be returned by the robots. These related objects make up a domain model, modeling the problem domain. Even if you only have a single object (as in this case), it still constitutes a domain model.

QUICK START TUTORIAL 1: DATA COLLECTION

15

Step 1 Start up ModelMaker

Now that we have designed the object model to use, the time has come to create it in ModelMaker. Action Start ModelMaker, e.g. by selecting it from the Start Menu (Windows systems). Effect ModelMaker will start and show the welcome screen shown in Figure 4.

Figure 4: ModelMaker Welcome Screen

16

QUICK START GUIDE

Step 2 Create a New Domain Model

Now create a new domain model. Action In the welcome screen, select Create a new domain model and click OK. Effect The ModelMaker window will appear as shown in Figure 5.

Attribute Table Object List

Add, Remove, and Configure Attributes

Add, Remove, and Preview Objects

Figure 5: ModelMaker Main Window

QUICK START TUTORIAL 1: DATA COLLECTION

17

Step 3 Create the Tip Object

Now we need to add an object to our domain model. To do this, click on the button in the lower left corner of the window. Select Output Object from the list and click OK.

Figure 6: Creating a New Object The new object now appears in the list of objects to the left. Note that it is currently unnamed. Set the name of the object to Tip by editing the Name field.

Step 4 Define Attributes for the Tip Object

Below the name of the object is an empty list of attributes in the object. Now we will add the four attributes to hold information about the tip.

18

QUICK START GUIDE

Press the

button just below the list of attributes.

Figure 7: Adding an Attribute Write title as the Name of this attribute, and select Short Text as its Type. When you have done this the dialog should as shown in Figure 7. Then press OK. You will see the new attribute appearing in the list of attributes.

QUICK START TUTORIAL 1: DATA COLLECTION

19

Now add the three other attributes. They should be declared so the Tip object type contains the four attributes shown in Figure 8. Attribute Name title number url description Type Short Text Integer Short Text Long Text

Figure 8: Attributes of the Tip Object When you have added all four attributes, your ModelMaker window should look like this:

Figure 9: ModelMaker after Addition of Attributes

20

QUICK START GUIDE

Step 5 Validate the Domain Model

Now that the model is complete, we should make sure that the Domain Model is valid. Press the button in the toolbar. The following message box should appear:

Figure 10: Validating the Domain Model If you do not get this dialog, but instead an error dialog, you need to go back and find out where things went wrong before you continue the tutorial.

Step 6 Save the Model

Now store the model to disk. Press the the name Tip for the domain model. button to save the model. Choose

QUICK START TUTORIAL 1: DATA COLLECTION

21

Creating the Robot

Now the time has come to write the robot itself, using the RoboMaker application.

Step 7 Start RoboMaker

Start RoboMaker, e.g. by selecting it from the Start Menu (Windows systems). When RoboMaker has started, a welcome screen like the one shown in Figure 11 is opened.

Figure 11: RoboMaker Welcome Screen

22

QUICK START GUIDE

Step 8 Create a new Robot

We select Create a new robot and click OK. This will open the New Robot Wizard, which will assist us in writing the robot.

Step 9 Choose Robot Type

In the first wizard step, we select Data collection robot and click Next.

Figure 12: Choosing the Robot Type

QUICK START TUTORIAL 1: DATA COLLECTION

23

Step 10 Enter the URL to Start From

In the new wizard step, enter the URL to start from: www.kapowtech.com/tutorial/quickstart/index.html and click Next.

Figure 13: Entering the URL to Start From

24

QUICK START GUIDE

Step 11 Choose Output Object

In the new wizard step, we choose the output objects. The list of objects should look familiar to you by now. It uses the same basic design as all other lists in Kapow Mashup Server. You can add, remove, and change the order. Click to add a new object, and select an object type from the drop-down list, which says Select an Object. Choose the one named Tip, as shown in Figure 14.

Figure 14: Choosing Output Object Click Finish when you are done. The RoboMaker Main Window should now look similar to the one shown in Figure 15.

QUICK START TUTORIAL 1: DATA COLLECTION

25

Robot View

State View

Step View

Objects View

Figure 15: The Robot Notice that the robot already contains two steps. The first step loads the page, and the second step, which is the current step, has not yet been configured.

26

QUICK START GUIDE

Step 12 Loop Through Table Rows

We will now loop through all the table rows in the table in the middle of the screen. Action The loaded page contains a table. We want to loop through the rows of this, that is, look at them one by one and thereby extract each tip in the table. To do this, place the mouse at some cell in the table, e.g. Number, and right click. Choose Loops | For Each Table Row | Exclude First Row. The first row should be excluded because it contains the headers of the columns.

Figure 16: Looping Through Table Rows Notice the number (1) in the For Each Tag Path step (see Figure 17). This indicates that we are currently looking at the first row that the robot will iterate over.

QUICK START TUTORIAL 1: DATA COLLECTION

27

Step 13 Extract Tip Number

The next thing we want to do is to insert a step to extract the number of each tip. Action In the Browser View, right-click on the tip number, and select the menu item Extraction | Extract Number | Tip.number.

Figure 17: Extracting Tip Number

28

QUICK START GUIDE

A dialog for extracting a number now appears. Make sure the Convert to Integer checkbox is checked.

Figure 18: Extract Number Configuration Click OK to close the dialog. Have a look at the Objects View (in the lower right corner). In the Output Objects tab, you can see the output objects and the values of their attributes. You will now see that the number attribute of the Tip object, as expected, has the value 4.

QUICK START TUTORIAL 1: DATA COLLECTION

29

Step 14 Extract Tip URL

Similarly, we also want to extract the URL of the tip. Action In the Browser View, right-click on the title of the tip and select the menu item Extraction | Extract URL | Tip.url.

Figure 19: Extracting Tip URL

30

QUICK START GUIDE

Step 15 Load the Page with the Tip

To get further information about the current tip, we need to load the page containing the tip itself. So we insert a step to do this. Action In the Browser View, right-click on the title of the tip and select the menu item Click.

Figure 20: Loading the Page with the Tip

QUICK START TUTORIAL 1: DATA COLLECTION

31

Step 16 Extract Tip Title

Now, we want to extract the title and the description of the tip. We start by extracting the title. Action In the Browser View, right-click on the title of the tip and select the menu item Extraction | Extract Text | Tip.title.

Figure 21: Extracting Tip Title

32

QUICK START GUIDE

Step 17 Extract Description

Finally, we will extract the description of the tip. Action In the Browser View, right-click on the tip text in the Browser View, and select the menu item Extraction | Extract Text | Tip.description.

Figure 22: Extracting Tip Description

QUICK START TUTORIAL 1: DATA COLLECTION

33

Step 18 Return Object

The last thing we need to do to complete the robot is to tell it that the object we have extracted is complete, that is, we have extracted all its attributes. This is done by inserting a Return Object step.: Action From the Action tab in the Step View in the right part of the RoboMaker window, choose Return Object as the step action. Choose the Tip object as the object to return.

Figure 23: Returning Tip Object

Step 19 Save the Robot

Now the robot is complete. Save it by pressing the Name it TipOfTheWeek.robot. button in the toolbar.

It is important that you save the robot in the default location suggested by RoboMaker, as we will try to access it from other applications later.

Running the Robot

In the following steps, we will show various examples of how to run our TipOfTheWeek robot.

34

QUICK START GUIDE

Step 20 Run in RoboMakers Debugger

Open the RoboDebugger in RoboMaker by clicking on the button on the . You will see toolbar. When the debugger opens, run the robot by pressing a list of returned objects appearing as shown in the figure below. These are the objects that your robot has extracted. You may click on an object in the list and see its content in the panel below the list.

Figure 24: Running in RoboDebugger

Step 21 Run Using RoboRunner

Now, let us run the robot using RoboRunner. Open a command prompt and go to a folder where you would like the output files from RoboRunner to be placed (this can be any folder, and these sample output files can safely be deleted afterwards). In the command prompt, enter the following: RoboRunner -SimpleRobotInfoEnvironment -FileStorageEnvironment fileName=objects.txt -FileMessageEnvironment fileName=messages.txt -file "C:/Program Files/Kapow Mashup Server 6.4/Project/Library/TipOfTheWeek.robot"

QUICK START TUTORIAL 1: DATA COLLECTION This runs the robot and places the extracted objects in the file objects.txt and messages in the file messages.txt. Open these files, e.g. with a text editor, to see the contents of the objects that were extracted and the messages given.

35

Step 22 Start RoboServer

Now, let us run our robot using RoboServer. To do this, you must first start RoboServer. On Windows, RoboServer can be started from the Start menu. On Linux and UNIX variants, RoboServer is started with the command: RoboServer service socket:50000 This will start RoboServer listening on a socket at port 50000.

36

QUICK START GUIDE

Step 23 Start RoboClient

First, let us try to communicate with RoboServer using RoboClient, so start RoboClient.

Step 24 Run the Robot from RoboClient

We first select a protocol for communication with RoboServer. In the RQL Protocol property, select Socket, and in the RQL Request property, select the Execute Request. Then set the Robot URL to library:/TipOfTheWeek.robot and press the button to send the request. The responses will appear in the list to the right as shown in Figure 25.

Figure 25: Running from RoboClient

QUICK START TUTORIAL 1: DATA COLLECTION

37

Figure 26 shows the received responses as a list. As you can see the robot sends back four extracted objects containing the tips.

Figure 26: RQL Responses

Step 25 Run the Robot from the RoboSuite Java API

This part of the guide shows how to run the robot using the RoboSuite Java API. It assumes that you have a Java SDK installed. Feel free to skip this step if you are not planning to write Java applications accessing RoboServer.

38

QUICK START GUIDE

The code example is written for a Windows platform. If you are running Linux or a UNIX variant, you need to replace all semi-colons with colons in the text below. Open a command prompt in the installation folder. Then go to the subfolder API/robosuite-java-api/examples by typing the following two commands: cd API cd robosuite-java-api cd examples Compile the RunRobot.java example as follows (on UNIX you will have to replace the semi-colons with colons in the classpath below): javac -classpath ../lib/robosuite-api.jar;../lib/crimson.jar RunRobot.java Now run the example (on UNIX you will have to replace the semi-colons with colons in the classpath below): java -cp .;../lib/robosuite-api.jar;../lib/crimson.jar RunRobot library:/TipOfTheWeek.robot This will run the robot and output the results to the screen. Take a look at the Java code in RunRobot.java to learn how the API was used to run the robot.

Step 26 Run the Robot from the RoboSuite .NET API

This part of the guide shows how to run the robot using the RoboSuite .NET API. It assumes that you have the Microsoft .NET Framework SDK version 2.0 installed on your computer. 1 Open a command prompt in the API/robosuite-dotnet-api/examples directory of your installation directory. Run the build.bat file to compile the C# source files in this directory. On successful compilation you will get a number of .exe files ready to run. Now type: RunRobot.exe library:/TipOfTheWeek.robot This will run the robot and output the result to the screen. If you look at the code in RunRobot.cs you can see how the API is used to run the robot.

1 The Microsoft .NET Framework SDK version 2.0 can be downloaded from Microsofts web site.

QUICK START TUTORIAL 1: DATA COLLECTION

39

Summary
This tutorial described how to create a simple model and robot, and how to run the robot in a variety of ways. Through RoboClient you have gotten your first view of RQL requests and replies, and perhaps you have run your first robot through the RoboSuite Java API or .NET API.

40

QUICK START GUIDE

Quick Start Tutorial 2: Clipping

This chapter contains a short tutorial on clipping. It will guide you through the creation and execution of a simple clipping robot for use as a portlet in a portal. It should take about 30 minutes to complete. The goal of the tour is to introduce you to the basic clipping concepts and show you an example of how to deploy a clipping robot.

Overview
We will be creating a simple clipping robot that is used for transferring the search functionality of Google into a portlet. Note that Google may change over time, so you may not experience exactly the same results as shown in the figures. Clipping is enabled in collaboration between RoboServer and a web server. In order to get full benefit from this tutorial, you will therefore need to have a web server available that is supported by Kapow Mashup Server. This tutorial includes descriptions of how to install and set up a web server and how to deploy the clipping robot.

Creating the Clipping Robot

We start by creating the clipping robot, using the RoboMaker application.

Step 1 Start RoboMaker

If RoboMaker is not already running, start it now, e.g. by selecting it from the Start Menu (on Windows systems).

QUICK START TUTORIAL 2: CLIPPING

41

Step 2 Create a New Robot

If you have just started RoboMaker, select Create a new robot in the welcome screen and click OK. Otherwise, press the icon to open the New Robot Wizard.

Figure 27: Creating a New Robot

42

QUICK START GUIDE

Step 3 Choose Robot Type

In the first wizard step, select Clipping robot and click Next.

Figure 28: Choosing Robot Type

QUICK START TUTORIAL 2: CLIPPING

43

Step 4 Enter the URL to Start From

In the next wizard step, enter the URL to start from: www.google.com and then click Finish to create the robot.

Figure 29: Entering the URL to Start From

44

QUICK START GUIDE

The RoboMaker Main Window should now look similar to the one shown in Figure 30.

Figure 30: The Clipping Robot A simple robot has been created, consisting of four steps. The first step is called Begin Clip, and the remaining steps form the default clip branch of the robot. The default clip branch is the branch that will be used for clipping all pages that you do not define specific clip branches for. Also notice that the Portlet View has opened. The Portlet View shows your clips as they will appear in the portlet. You can interact with the clips in the same way as a user interacts with the portlet. By default, the Portlet View opens and closes automatically as needed. However, you can also open and close the Portlet View yourself by clicking the icon in the Main Window of RoboMaker. As a rule of thumb, you should always use the Portlet View when navigating a clipping robot. Within the current clip branch, however, you can change steps as usual.

Step 5 Configure the URLs to Clip

The next thing we want to do is to configure which URLs the robot should clip. We want all pages on Google itself to be clipped and be shown inside the

QUICK START TUTORIAL 2: CLIPPING

45

portlet, but all links away from Google such as when the user clicks a link in a search result should open in a new window and not be clipped. This way, the users of your portal will stay in your portal as long as they navigate on Google, but any links away from Google will open in new windows separately from your portal. To do this, click the button in the Portlet View to open the Robot Configuration Window.

Figure 31: Robot Configuration Window In the Clipping Restrictions property, select Include Only URLs in these Domains. In the Domains property that appears, enter *.google.* to include all links on Google. Make sure the Excluded Links property is set to Open in New Window, so that other links will open in new windows. Click OK to close the Robot Configuration Window. The clipping robot that you have now is a fully functional clipping robot, and you can skip forward to step 12 if you want to try the robot as it is.

46

QUICK START GUIDE

Alternatively, you can continue the tutorial and see how you can make modifications to the Google pages as part of the clipping.

Step 6 Add a New Clip Branch

We want to modify how the start page of Google looks in the portlet. To do this, first add a new clip branch specifically for this page. In the Portlet View, icon to open the Edit Clip Wizard: click the

Figure 32: Adding a New Clip Branch Make sure the Add new clipping rules option is selected, and click Next to continue.

QUICK START TUTORIAL 2: CLIPPING

47

Step 7 Enter Name of New Clip Branch

On the next wizard page, enter a name for the new clip branch, or use the suggested name.

Figure 33: Entering the Name of the New Clip Branch Click Finish to add the new clip branch. You will now be placed at the location in the robot where you can configure the new clip branch. The Portlet View will close automatically to allow you to use the Main Window for this.

48

QUICK START GUIDE

Step 8 Hide a Button

The first modification is to hide the search button called Im Feeling Lucky. In the Browser View, right-click on the Im Feeling Lucky button and choose Hide Tag, as shown in Figure 34.

Figure 34: Hiding a Button

QUICK START TUTORIAL 2: CLIPPING

49

Step 9 Select the Logo

The next modification is to select the parts of the page to clip. This is done by configuring the Clip step, so ensure that you are on the Clip step click on it if not. In the Browser View, click on the Google logo, so that a green box appears around it. If the green box covers only part of the logo, click the icon in the toolbar above the Browser View to expand the box until it covers the logo. If the box becomes too big, click the icon to reduce it. When the green box covers the logo, right-click in the box and select Use only this Tag, as shown in Figure 35.

Figure 35: Selecting the Logo This will select the Google logo as one of the tags to clip from the page.

50

QUICK START GUIDE

Step 10 Select the Search Form

Then, in the Browser View, click the text input field (the search field). In the toolbar above the Browser view, click the icon to expand the green box until it covers the search form and the links above it (see Figure 36 below). If the box becomes too big, click the icon to reduce it. When the box covers the form and the links, right-click in the box and select Use this Tag (not Use only this Tag), as shown below:

Figure 36: Selecting the Search Form Now, you have selected the Google logo and the search form as the two parts of the page to clip, leaving out the rest of the page.

QUICK START TUTORIAL 2: CLIPPING

51

Step 11 Show the Clip

Click the icon to the right in the toolbar at the top of the RoboMaker Main Window. This will open the Portlet View and show the clip that you have defined:

Figure 37: The Modified Clip

52

QUICK START GUIDE

Step 12 Make a Search

Now, try out your portlet in the Portlet View by entering some text (e.g. Kapow Technologies) in the search field and clicking the Google Search button or hitting Enter.

Figure 38: Entering a Search Text

QUICK START TUTORIAL 2: CLIPPING

53

The Portlet View shows the search results as they will appear in your portlet:

Figure 39: The Search Results icon to restart the clipping, Try performing a new search by clicking the entering another text, and clicking the search button again. Notice how the robot is being executed in the Main Window of RoboMaker as you are navigating in the Portlet View. For the first page, the clip branch that you have added will be executed. For the search results, the default clip branch will be executed.

Step 13 Ensure the Correct Clip Branch is Used

Make sure you are on the search results page in the Portlet View, by making a new search if necessary. Click on the small Google logo in the upper left corner to go back to the front page. Check whether the front page is clipped as you wanted it to be, with only the logo and the search form included. If not, this happens because the link back to the front page uses a different URL than the standard one (www.google.com) to get to the front page. To fix

54

QUICK START GUIDE

this, click the icon in the toolbar of the Portlet View. In the Edit Clip Wizard, select Clip using other rules:

Figure 40: Clipping Using another Clip Branch

QUICK START TUTORIAL 2: CLIPPING

55

Click the Next button. On the next page, select the clip branch that you want to use for this page this is the clip branch that you added, and which is already selected by default:

Figure 41: Selecting the Clip Branch to Use Click Finish. The Edit Clip Wizard will now configure the robot to ensure that the selected clip branch will be used for the page. As you can see in the Portlet View, the front page is now clipped as desired. If you want to, verify this by making another search in the Portlet View, and clicking on the Google logo again on the results page to get back to the front page.

Step 14 Save the Robot

The robot is now complete, so save it by pressing the button in the toolbar. Name it GoogleClip.robot. Do not close RoboMaker yet. The rest of this tutorial will show you how to deploy and run your clipping robot.

56

QUICK START GUIDE

Installing Tomcat
Next, we install the Apache Tomcat web server. You can skip this section if you already have Apache Tomcat installed. If you do not want to deploy your robot, or want to deploy it on another web server, go to the Summary section at the end of this chapter.

Step 16 Download Apache Tomcat

The Kapow Developer Connection contains a link to the binary distribution of the Apache Jakarta Tomcat web server. Windows users should use this link to get the latest Tomcat installer: http://kdc.kapowtech.com/tomcat.html Users of other systems can go to the Apache Jakarta Projects homepage for a list of downloads. http://jakarta.apache.org

Step 17 Install Apache Tomcat

Choose to install the program in C:\apache-tomcat folder on a Windows system, or /apache-tomcat folder on Unix-like systems. Press Next after choosing a folder.

Figure 42: Selecting the Tomcat Installation Folder

QUICK START TUTORIAL 2: CLIPPING

57

Then choose an administrator password for Tomcat. This isn't really important for this tutorial, so just choose an arbitrary password, and click Next.

Figure 43: Choosing Administrator Password

58

QUICK START GUIDE

After choosing the password, press Next, followed by Install, and Tomcat will complete the installation. Press Finish to finish the installer.

Figure 44: Tomcat Installation Completed

Step 18 Install the RoboSuite Tag Library Demo Web Application on Apache Tomcat
To create the clipping portlet, we need a web application to place it in. The RoboSuite JSP Tag Library Demo Web Application is an excellent candidate for this. Copy the file API\robosuite-jsp-taglib\examples\rstl-demo.war located in the installation folder to the C:\apache-tomcat\webapps folder. When you start Tomcat in the next step, the application will be deployed automatically.

Step 19 Test that Tomcat is Running Correctly

Start Apache Tomcat. The installation provides you with a shortcut to start Tomcat. On Windows systems double click on the quick launch icon and press Start to start a Tomcat server. On non-Windows systems, you should use the startup.sh script in the bin folder where you installed Tomcat. Go to the URL http://localhost:8080/rstl-demo/. This should show you the introduction page to the RoboSuite Tag Library Demo Web Application.

QUICK START TUTORIAL 2: CLIPPING

59

Deploying the Clipping Robot

Now we can deploy the clipping robot, so go back to RoboMaker (if RoboMaker is no longer running, restart it and load the GoogleClip.robot).

Step 20 Start the Java Servlet Clip Wizard

In the Clip menu of the RoboMaker main window, choose Standard Java | Java Servlet Clip Wizard. This will open the following window.

Figure 45: JSP Clipping Wizard Browse to the root folder of the web application. In our case, the path is C:\apache-tomcat\webapps\rstl-demo. Click the Next button.

60

QUICK START GUIDE

Step 21 Choose a Name for the Deployment Descriptor File

In the next screen, you choose the name for the Deployment Descriptor file (including its path from the root of the web application) that the wizard will generate. The wizard will install a servlet and the URL pattern *.clip is mapped to this, so a HTTP request that hits a Deployment Descriptor file will be redirected to this servlet which will send back the clip. Just use the default file name suggested by the wizard and click the Next button.

Figure 46: Choose a Name for the JSP File

Step 22 Finish the Wizard

The three next pages contain advanced settings related to deployment, and other further configuration properties, which we do not need right now. The only thing you need to know is that the protocol that the clip will use when contacting the RoboServer is a socket-based protocol at port 50000 on the local machine. Just click Finish to make the wizard perform the requested operations.

QUICK START TUTORIAL 2: CLIPPING

61

When the wizard has completed, you can view a short summary of what the wizard has done by clicking Details.

Figure 47: Details of the Wizard Execution Click OK twice to close the two dialogs.

Running the Clipping Robot

Step 23 Restart Tomcat
The wizard has added some library files and updated the web.xml of the web application, so we need to restart Tomcat. Some web servers can auto-detect such changes, but not Tomcat. icon and In Windows, you restart Tomcat by double-clicking on the selecting Stop followed by Start. On non-Windows systems, you shut down Tomcat by closing its process, or invoking the shutdown.sh script in the bin folder of the Tomcat installation folder. Restart Tomcat by invoking the startup.sh script.

Step 24 Start RoboServer

Start RoboServer. This can be done from the Start Menu on Windows Systems, or by invoking the RoboServer executable file in the bin subfolder of the installation.

Step 25 Try out Clipping

Go to the following page, to try out your clipping robot: http://localhost:8080/rstl-demo/GoogleClip.clip.

62

QUICK START GUIDE

Summary
This tutorial introduced you to clipping by building and running a simple clipping robot.

Next Steps
If you want to deploy your clipping robot on another web server than Apache Tomcat, you can use the portlet generation wizards available in the Clip menu in the RoboMaker Main Window. For more information on using the wizards, refer to the online documentation for the wizards, or the Code Generation Guide. If there is no portlet generation wizard available for your target web server, please contact Kapow Technologies for further information. For more information about clipping robots, see the How to Clip chapter in the RoboMaker Users Guide. You can also go on and try the various tutorials found in the ModelMaker and RoboMaker Users Guides, or start on the recommended reading.

INDEX

63

Index
.
.NET, 7

D
Data Collection, 15 deploying a clipping robot, 62 Deployment Descriptor file, 63 domain model, 16, 22 domain model validating a, 22

A
administrator guides, 10 Apache Jakarta Tomcat, 59 attribute, 19 extracting, 33 type, 20

E
extracting attributes, 29, 33 extracting numbers, 29 extracting text, 33

B
BEA, 7 BEA Java Web Service Wizard, 13 BEA Workshop Control Developer's Guide, 10

H
HTML, 2

C
C#, 7 Click step, 32 clipping, 42 Edit Branch Wizard, 49 hiding tags, 51 Portlet View, 47 restricting links, 48 selecting tags to clip, 52 start URL, 45 using another clip branch, 56 Code Generation Guide, 9 Control Center, 7 Control Center User's Guide, 9 converting to integer, 30 creating robots, 6

I
installation, 1

J
Java, 7 Java Server Pages, 7 JavaScript, 2 JSP, 7

K
Kapow Developer Connection, 1 Kapow Mashup Server, 2 KDC, 1

L
looping, 28

64

QUICK START GUIDE RoboRunner, 7, 36 running a robot in, 36 RoboRunner User's Guide, 9 RoboServer, 4, 7, 37, 64 RoboServer User's Guide, 9 RoboSuite .NET API, 7, 40 RoboSuite .NET Programmer's Guide, 10 RoboSuite Control, 7 RoboSuite installation folder, 15 RoboSuite Installation Guide, 10 RoboSuite Java API, 4, 7, 40 RoboSuite Java Programmer's Guide, 9 RoboSuite JSP Tag Library, 7 RoboSuite JSP Tag Library Demo, 61 RoboSuite JSP Tag Library Programmer's Guide, 9 RoboSuite Tag Library, 4 robot clipping creating a, 42 configuration, 48 creating a, 23 running a, 36 saving a, 35, 58 type, 44 Robot Configuration Window, 48 robots, 5 creating, 6 running, 7 RQL Service User's Guide, 10 Running a robot from RoboClient, 38 running a robot from RoboSuite .NET API, 40

M
managing robots, 8 ModelMaker, 6, 17 ModelMaker User's Guide, 9

O
object returning, 35 object model, 5 objects, 5 output object, 19

P
Portlet View, 47 programmer guides, 9

R
recommended reading, 10 clipping, 11 data collection, 11 integration .NET, 13 Java, 12 web applications or portals, 12 Kapow Mashup Server basics, 11 web services Java, 13 return object, 35 returning object, 35 RoboClient, 7, 38 RoboDebugger, 36 RoboMaker, 6, 23, 42 RoboMaker User's Guide, 9 RoboManager, 8 RoboManager User's Guide, 9

INDEX running a robot from RoboSuite Java API, 40 running a robot in RoboRunner, 36 running robots, 7 running robots in RoboDebugger, 36

65

U
URL extraction, 31 user guides, 9

W
Web Integration, 2 Web Service Wizard, 13 WebLogic, 7 Workshop, 7

T
Tomcat, 59