EAI Concepts & TIBCO

Objective of EAI


Objective of EAI is to connect disparate systems in a cost-efficient fashion. Such integration is possible by employing a set of enabling techniques such; the key techniques include loose coupling or the reduction of topological by introducing a common medium between the to-be integrated systems (integrate once, recompose many times). Specific integration cases aside (such as batch integration, ETL for data warehousing, common database / operational data stores, etc), EAI vendors offers a generic model based on message-oriented middleware (MOM). This model includes the following functions : To-be-integrated application Connectivity between applications and transport Transport Transformation & Routing Task Flow Business Process Management

Integration Stack

Integration Stack
Connectivity The Connectivity function encapsulates the transport and target application interface by connecting each integrated application to the transport function. In the EAI world, the connectivity function is provided by adapters and connectors. Transport Transport performs two important functions: * Moves the data across the network.

* Facilitates some of the key aspects loose coupling, most importantly temporal and technological de-coupling.

Transformation & Routing

In almost all integration activities, transformation from source format to target format is required. The base transformation process involves format modifications (both syntax and semantic mapping). More sophisticated transformation services offer message augmentation (message boosting) or data cleansing (for example removal of duplicates) that may be required to handle the differences between the to-be-integrated applications.

Integration Stack
Except for basic point-to-point integration routing of messages is a key component of an integration solution. These message routing activities encompass: Managing the delivery of the messages over communication connections, including protocol conversion, flow control, guaranteed delivery, and connection optimization (for example, connection pooling) Multi-point message decomposition/re-composition, enabling one to many and many to one message routing Content-based routing with associated directory-based or rules-based routing. Transformation and routing function are typically provided by the same component of an integration broker and for that reason we group them together. Task Flow The Task Flow Management function of the broker coordinates relatively simple, short time activities amongst the integrated systems. Task flow management allows for re-combining applications functionality to yield a more complex functionality.

Integration Stack
Frequently, the Business Process Management function delegates a single business tasks to the Task Flow Management function. The Task Flow Management function service translates the business task into a set of lower-level (often application specific) technical tasks. Task Flow Management service is also known as: technical process management, micro-workflow, system-level workflow, and message choreography

Business Process Management The Business Process Management function (BPM) coordinates long-running business processes. In many aspects, BPMS resembles Task Flow Management function. BPM differs from Task Flow Management in that: It is geared towards managing tasks that may range from hours to months in duration. BPM persists its state in a database. BPM focuses on business-level tasks, frequently tasks that are performed by humans. To support such tasks, BPM support sophisticates access control and authorization models, escalation procedures, task delegation, etc.

TIBCO Implementation


TIBCO started its presence in the EAI space started as a messageoriented middleware vendor with its flagship product, RendezVous (RV). Over time, TIBCO shifted its focus from MOM to the higher stack levels. Either by acquisitions (InConcert, Staffware, Talarian, BAM platform) or in house development (MessagBroker, IntegrationManager, BusinessWorks), TIBCO has assembled a complete EAI stack

Introduction to Rendezvous Software

Rendezvous software suite includes two main componentsa Rendezvous programming language interface (API) and the Rendezvous daemon.

The Rendezvous daemon runs on each participating computer on your network. All information that travels between program processes passes through the Rendezvous daemon as the information enters and exits host computers. The daemon also passes information between program processes running on the same host.

Decoupling and Data Independence Rendezvous software allows looser coupling between the components of a distributed system. Loose coupling decreases costs for development, operation and maintenance, and increases system longevity. Rendezvous self-describing data messages promote data independence; producers and consumers of data can communicate even if they do not share the same internal representations for data. Communicating programs can run on different hardware architectures, even though they use different bit order, byte alignment or numeric representations. Location Transparency Rendezvous software uses subject-based addressing technology to direct messages to their destinations, so program processes can communicate without knowing the details of network addresses or connections. Subject-based addressing conventions define a uniform name space for messages and their destinations.

Introduction to Rendezvous Software

Architectural Emphasis on Information Sources and Destinations
Decoupling distributed components eliminates much of the complexity traditionally associated with network programming. Rendezvous software frees you to devote more resources to solving application problems.

Reliable Delivery of Whole Messages Rendezvous software provides reliable communications between programs, while hiding the burdensome details of network communication and packet transfer from the programmer. Rendezvous software takes care of segmenting and recombining large messages, acknowledging packet receipt, retransmitting lost packets, and arranging packets in the correct order. You can concentrate on whole messages, rather than packets.

Rendezvous Operating Environment

Networking Basics
To better understand the concept of RV, lets go thru some of the Networking Basics

Connection-Oriented and Connectionless Protocols



Connection-Oriented Protocols: These protocols require that a logical connection be established between two devices before transferring data. This is generally accomplished by following a specific set of rules that specify how a connection should be initiated, negotiated, managed and eventually terminated. Usually one device begins by sending a request to open a connection, and the other responds. They pass control information to determine if and how the connection should be set up. If this is successful, data is sent between the devices. When they are finished, the connection is broken. Example : tcp/ip, ftp, telnet Connectionless Protocols: These protocols do not establish a connection between devices. As soon as a device has data to send to another, it just sends it. Example : udp

Networking Basics
Unicast, Broadcast and Multicast Messages


Unicast Messages: These are messages that are sent from one device to another device; they are not intended for others. Broadcast Messages: As the name suggests, these messages are sent to every device on a network. They are used when a piece of information actually needs communicating to everyone on the network, or used when the sending station needs to send to just one recipient, but doesn't know its address. Multicast Messages: These are a compromise between the previous two types: they are sent to a group of stations that meet a particular set of criteria. These stations are usually related to each other in some way, such as serving a common function, or being set up into a particular multicast group. (Note that one can also consider broadcast messages to be a special case of multicast, where the group is everyone.

Networking Basics
TCP and UDP


Transmission Control Protocol (TCP): A full-featured, connectionoriented, reliable transport protocol for TCP/IP applications. TCP provides transport-layer addressing to allow multiple software applications to simultaneously use a single IP address. It allows a pair of devices to establish a virtual connection and then pass data bidirectionally. Transmissions are managed using a special sliding window system, with unacknowledged transmissions detected and automatically retransmitted. Additional functionality allows the flow of data between devices to be managed, and special circumstances to be addressed. User Datagram Protocol (UDP): A very simple transport protocol that provides transport-layer addressing like TCP, but little else. UDP is barely more than a wrapper protocol that provides a way for applications to access the Internet Protocol. No connection is established, transmissions are unreliable, and data can be lost.

TRDP TIBCO Reliable Datagram Protocol



RV runs on TRDP TIBCO Reliable Datagram Protocol. It is implemented by rvd. TRDP is built on top of UDP and adds a sequence number to each outbound packet. TRDP uses buffer to store outbound messages for specific interval (Typically 60 sec.) If message packet is lost, the consuming rvd requests retransmission. In contrast to TCP/IP which is pessimistic, TRDP is optimistic: it assumes messages will arrive at their destination. If the message is lost, then TRDP sends negative acknowledgement (NAK) requesting retransmission. If stronger delivery assurance is required, RV Certified (RVCM) or RV Transactional (RVTX) is used. RVCM uses ledger files at both Publisher and Subscriber ends. These ledger files stores messages, and these messages sit in ledger file until it gets confirmation from all receivers.

Messages and Data



Computer hardware and operating system platforms use different conventions for data representation. Data from one platform can be unintelligible on another platform. Rendezvous software uses a unified data representation to exchange messages among all supported platforms. Messages are the common currency that Rendezvous programs use to exchange data. Rendezvous messages contain fields of self-describing data. Every message has a subject name, which describes its destination. All data that enters or leaves a program through the Rendezvous daemon must be encapsulated in the fields of a message. As an abstraction, a message is a collection of self-describing data fields, which travel together between programs, processes or threads.

Fields
 

Each field contains one data item of a specific datatype. Programs can identify and access the individual fields of a message either by name or by numeric identifiers. From the programmers point of view, a message is a set of fields. Programs manipulate messages using API calls. A program can create a message, add fields to it, remove fields from it, get a field from a message, update the data value in a field, and destroy a message.

Wire Format


At a lower level, beyond these abstract operations, each message exists as a byte sequence in Rendezvous wire formata uniform representation suitable for network communication among diverse hardware, operating system, and programming language platforms. Programs never access this representation, yet it is the foundation for all Rendezvous communication.

Subject-Based Addressing and Message Destinations

Subject Names  Subject names consist of one or more elements separated by dot characters (periods). The elements can be used to implement a subject name hierarchy that reflects the structure of information in an application system. These strings are examples of valid subject names: RUN.HOME RUN.for.Elected_office.President Wildcard Subject Names  A program can receive a group of related subjects by listening for a wildcard subject.

Multicast and Point-to-Point Messages

Point-to-Point Messages  A point-to-point message has at most one recipient program; its destination is an inboxa subject name created dynamically by a specific program process

Multicast Messages  A multicast message is any message with many potential recipients. Potential recipients are called subscribers.

Transport


The software mechanism for sending and delivering messages is called a transport. A transport defines the delivery scopethat is, the set of possible destinations for the messages it sends.

Network Transport Parameters

 

Several independent distributed applications run on the same network, and you must isolate them from one another (service parameter). Programs use the Rendezvous routing daemon, rvrd, to cooperate across a WAN with programs that belong to a particular service group, and the local programs must join the same service group (service parameter). A program runs on a computer with more than one network interface, and you must select a specific network for outbound multicast Rendezvous communications (network parameter). Computers on the network use multicast addressing to achieve even higher efficiency, and you must specify which multicast groups to join (network parameter). A program runs on one computer, but connects with a Rendezvous daemon process running on a different computer, and you must specify the remote daemon to support network communications (daemon parameter). Two programs use direct communication. Both programs must enable this feature and specify its service (service parameter).

Workshop 1
Try 2 commands ..tibrvsend and tibrvlisten cd C:\tibco\tibrv\bin C:\tibco\tibrv\bin>tibrvsend tibrvsend [-service service] [-network network] [-daemon daemon] <subject> <messages> C:\tibco\tibrv\bin>tibrvlisten tibrvlisten [-service service] [-network network] [-daemon daemon] subject_list

Workflow / Transformation using BusinessWorks



TIBCO BusinessWorks is a scalable, extensible, and easy to use integration platform that allows you to develop integration projects. TIBCO BusinessWorks includes a graphical user interface (GUI) for defining business processes and an engine that executes the process. TIBCO BusinessWorks also works with TIBCO Administrator, a web-based GUI for monitoring and managing run-time components.

Empty BW Project

Project Tree in Project Panel



The Typical project consists of the following sections AESchemas Process Shared Connections Utility Services Deployment

The folder structure is decided during the design phase of TIBCO based project and the same template is shared across interfaces.

Global Variables
 Global Variables these

variables allow you to specify constants that can be used throughout the project. The constants can be specified and changed while designing and testing your project. You can also specify different values for each deployment of your project.

Palettes


Palettes organize resources and allow you to add them to your project. You select resources in the palette panel and drag and drop them into the design panel to add them to your project. Palettes are grouped in different types FTP General HTTP JMS Java Process RV XML WSDL SOAP Transaction

Design Panel

 

The design panel shows the detailed design, above picture depicts sample workflow. You can drag and drop palettes from Palette Panel to Design panel to design the workflow.

Configuration Panel

The configuration panel allows you to specify various configuration options for each resource. The type and the purpose of the selected resource determine the contents of the configuration panel. Usually there are one or more tabs in the configuration panel that allow you to access the various configuration options. The tabs organize the configuration options for the resource.

Processes

Process definitions consist of these components:  Activities  Transitions  Groups  Shared Configuration Resources  Subprocesses

Activities


Activities are the individual units of work in a process definition. Activities are generally operations that interface to external systems, but activities can also perform internal processing. Activities are available on the various palettes in TIBCO Designer. Each palette has a set of activities that can be performed for that palette.

Activities

Mapping between activities

Transitions

Grouping Activities

Workshop2
        

Create a Process Use RV Publisher and RV subscriber activities. Use RV subscriber as Startup activity Listen on Subject Input Use transition to send this message to RV publisher activity. Publish the same message using RV publisher on subject Output. Use tibrvsend to publish a sample message DCOTG on subject Input. Use tibrvlisten to listen on Output subject. Go to test mode and test this sample application.

Observe Observe the directory structure of your project