Distributed Computing Fundamentals And Distributed Computing paradigms

Distributed System
Collection of independent computers, interconnected via a network, that are capable of collaborating on a task. Computers are considered independent if they do not share the memory or program execution space. Loosely coupled configuration

Distributed Computing
It is computing performed in distributed system. Web based applications are good examples of distributed computing
Network Services.
Email, ftp, telnet etc.

Network applications
Enterprise application such electronic auction etc. as online shopping,

Historically started with the development of Internet

Different Forms of Computing

Monolithic Computing
Performed on PCs known as single user monolithic computing Mainframes providing timesharing concept to support multiple users

Distributed Computing
Performed among multiple network connected computers, each of having its own processor and resources

Parallel Computing
Another form of distributed computing which uses more than one processor simultaneously to executes a single program

Different Forms of Computing

Cooperative Computing
Applied to projects such as SETI (Search for Extraterrestrial Intelligence) and distributed.net Parcels out large scale computing to workstations on Internet making use of surplus CPU cycles

Strengths and Weaknesses

Strengths:
The affordability of computers and availability of network access Resource sharing Scalability Fault tolerance

Weaknesses:
Multiple points of failure Security concerns

Prerequisite
Operating System
Process, Process states Concurrent Programming Concurrent processes running on multiple computers
Programming support is needed for communication among processes

Concurrent processes running on single computer

Supported by operating system

Concurrent programming in a process Process Vs. Threads

Network
Architecture, addressing, protocols etc.

Software Engineering
Procedural Vs. Object Oriented programming Modeling Language like UML

Prerequisite
Architecture of Distributed Applications
Three Layers
Presentation Application Logic Service Layer

Toolkits, Frameworks and Components

Toolkits or frameworks are collections of classes, tools and programming samples. Examples, JDK, .NET Components : EJB, COM, DCOM
Supports evolution of application by assembly of reusable components

Interprocess Communication
Backbone of the Distributed Communication Provides the ability for separate independent processes to communicate among themselves to collaborate on task Two independent processes running on different machines exchanges the data Unicast : One sender, One receiver Multicast : One sender, Multiple receiver IPC in form of API (Application Program Interface) provides abstraction of the details and intricacies of the system level facilities

IPC Program Interface

Send :
issued by sending process for the purpose of transmitting data. Allows identification of receiving process and specified the data

Receive :
Issued by receiving process to receive the data. Allows the identification of sending process and specified the space to store the data

IPC Program Interface

Connect :
For connection oriented IPC, allows the logical connection to be established between processes. One process issues a request-to-connect and other issues a accept-connection

Disconnect:
Previously created connection is deallocated on both the sides

IPC Program Interface

Issuance of operation causes the occurrence of an event
Send results into data transmission to the receiver Receive results into the delivery of data

Example:
Communication between Web-server and
Accept connection Receive (Request) Send (Response) Disconnect

Web-browser
Make connection Send (Request) Receive (Response) Disconnect

Event Synchronization
Major difficulty is that processes runs independently HTTP needs that two sides involved in communication must issue IPC operations in specific order Event synchronization is provided by blocking, suspending the process until operation issued by process has been completed Blocking operations are also called synchronous operations Alternatively, IPC operations may be asynchronous or nonblocking Event synchronization is fundamental to distributed computing Example : Receive issued by web-browser is blocking, while send issued by web-server is non-blocking

Event Synchronization
Four different modes
Synchronous send and synchronous receive
Required when logic of both processes requires that data sent must be received before further processing can start
Process 2

Process 1

Blocking send starts Blocking send returns

Transparent acknowledgement provided by the IPC facility

Blocking receive starts

Blocking receive ends

Event Synchronization
Asynchronous send and synchronous receive
Appropriate when senders logic does not depend on the receiving of the data at the other end

Process 1

Process 2

Non Blocking Send

Blocking receive starts

Blocking receive ends

Event Synchronization
Synchronous send and asynchronous receive
Causes different scenarios for a protocol session. Depends on IPC implementation. Three scenarios: Data already arrived at the time when receive operation is issued. Immediate delivery Data requested by receive operation has not arrived; no delivery to the process. Receive process can use polling (loop) until data is received Data requested by receive operation has not arrived; When data arrives IPC facility notifies the process2.

Event Synchronization
Process 1 Process 2 Process 1 Process 2 Nonblocking receive issued and returned immediately

Blocking send issued

Transparent Acknowledge Provided by The IPC facility

Nonblocking receive issued

Blocking send issued

Process 1

Process 2 Nonblocking Receive issued And returned immediately Process is Notifed of the arrival of data

Blocking send issued

Transparent Acknowledge Provided by The IPC facility

Event Synchronization
Asynchronous send and asynchronous receive
Data retained by IPC facility and receiving process is notified or Receiving process may poll for the data
Process 1 Process 2
Nonblocking receive issued and returnd immediately Process is notified of the arrival of the data and the data may now be delivered to it

NonBlocking Send issued

Timeouts and Threading

Blocking may cause indefinite suspension Two possible solutions
Timeout, specified in a program Program may create a child process or thread to issue a blocking operation. Mainline process continues with other operation

Care must be taken to avoid the deadlocks, which causes the indefinite suspension

Data Representation
Different languages have different size of data types and different representation When heterogeneous systems are involved, there are three possibilities
Sending process converts to representation of receiving process before sending Receiving process converts the senders representation upon receiving Exchange the data in External Representation

Data marshaling is supported by IPC performing

Serializing the data structure Converting data values to external representation

JAVA supports object serialization

Data Encoding
General purpose distributed application needs platform independent scheme of encoding the data exchanged. Three network data encoding standard exists
External Data Representation (XDR) from sun ANS.1 (Abstract Syntax Notation Number 1) from OSI Supports data tagging Each data is encoded with type,length,value and optinal tag to specify the interpretation XML (Extensible Markup Language) <message> <to>students@da-iict.org</to> <from>mts@ldce.org</from> <subject> Interprocess communication</subject> <text> IPC is backbone of distributed computing </text> </message>

Protocols
Text based protocols
Data exchanged as stream of characters FTP, HTTP, SMTP are text based

Request-Response protocols
One side issues the request and awaits for the response from other side Examples : FTP, HTTP, SMTP

Event diagrams and sequence diagrams are used to document the details of communication

IPC Paradigms
Connection oriented Vs. Connectionless Abstractions:
Level of Abstraction IPC Paradigms Example RPC/RMI Unix Socket API Winsock Serial/parallel communication

Remote Procedure/Method Socket API Data Transmission

Abstraction
Idea of hiding the details or encapsulation We often use abstraction when it is not necessary to know the exact details of how something works or represented, because we can still make use of it in its simplified form. Getting involved with the detail often tends to obscure what we are trying to understand, rather than illuminate it. Abstraction plays very important role in programming because we often want to model, in software, simplified version of things that exist in the real world without having to built the real things.

Distributed Computing Paradigms

Paradigm is a pattern, example or model
Level of Abstraction
Object space, Collaborative applications Network Services, object request broker, mobile agent Remote Procedure Call / Remote Method Invocation Client-Server, peer-to-peer Message Passing

An Example System
Online Auctioning System System handles only one auctioned item per session. During each auctioning session, an item is open for bids placed by the auction participants. At the end the auctioneer announces the outcome

Distributed Computing Paradigms

Message Passing Paradigm
Basic approach to Interprocess communication Sender and receiver communicates by exchanging the messages Operations needed are send and receive. For connection oriented IPC, connect and disconnect are also needed Similar to File I/O Socket API is an example. Sender writes message to socket, while receiver extracts the message from socket

Distributed Computing Paradigms

Client-Server Paradigm
Well known approach for network applications Two processes have asymmetric role. Server process provides services, while client process accesses the services by making requests Operations for server are listen and accept the requests and client process to issue the requests and accept the response Examples : HTTP,FTP,DNS etc. Connection oriented Socket API, RPC and RMI provides notion of clients and servers

Distributed Computing Paradigms

Example system
For session control
As a server, participant waits to hear an announcement from auctioneer (1) when session starts, (2) whenever there is an update on the current highest bid, and (3) when the session ends. As a client, the auctioneer sends a request that announces the three types of events outlined as above

For accepting bids

As a client, a participant sends a new bid to a server As a server, an auctioneer accepts new bids and updates the current highest bid

Distributed Computing Paradigms

Peer-to-Peer Paradigm
Participating processes play equal role, with equivalent capabilities and responsibilities Each participant may issue request to another participant and gets response. Well-known example is peer-to-peer file transfer service provided by Napster.com Appropriate for instant messaging, peer-to-peer file transfers, video conferencing and collaborative tasks Well known examples: JXTA project and Jabber an XML based protocol

Distributed Computing Paradigms

Example System
A participant contacts the auctioneer directly to register for the auction Auctioneer subsequently contacts each participants to initiate the auction session, during which individual participants may obtain the status and submit the bid At the end, winning participant is notified by the auctioneer. Other may get the status by contacting the auctioneer

Distributed Computing Paradigms

Message System Paradigm
Message System or Message Oriented Middleware (MOM) is an elaboration of basic message passing system Message system acts as intermediary among separate independent processes Message system acts as a switch for messages through which processes exchange message asynchronously, in a decoupled manner. Sender deposits a message to message system, which forwards it to message queue associated with each receiver.

Distributed Computing Paradigms

Message System Paradigm

Sender Receiver
Message queue Message System

Distributed Computing Paradigms

Message System Paradigm (Two models)
Point-to-Point Message Model
Message system forwards message from sender to the receivers message queue System acts as depository and allows sender and receiver to be decoupled. Sender deposits the message in receivers queue and receiver extracts it from queue. Same as basic message passing system, but provides additional abstraction for asynchronous operations. Implementation of auction system is same as message passing system. Difference is that messages are passes through Message middleware.

Distributed Computing Paradigms

Publish/Subscribe Model
Each message is associated with a specific topic or event Application subscribes for the message for the event When the event occurs, the process publishes the message announcing the event or topic The middleware message system distributes the message to all subscribers Offers powerful abstraction for multicast or group communication. The publish operation allows process to multicast, while subscribe operation allows process to listen for multicast

Distributed Computing Paradigms

Example system with Publish/Subscribe Model
Each participant subscribes to a begin-auction event message The auctioneer signifies the beginning of the auctioning session by sending a begin-session event message Upon receiving the begin-auction event, a participant subscribes to an end-auction event message The auctioneer subscribes to message for new-bid events A participant wishing to place a new bid issues a new-bid event message, which will be forwarded to auctioneer The auctioneer issues an end-auction event message to inform participants about outcome. If desired additional message events can be added to allow participants to monitor the status of the auction

Distributed Computing Paradigms

Remote Procedure Call Paradigm
Message passing is well for network protocols More abstraction is required for network programming so that programmer can built the distributed software similar to conventional application RPC provides such an abstraction Message passing is data oriented, while RPC is action oriented.

Distributed Computing Paradigms

Example system with RPC
The auctioning program provides a remote procedure for each participant to register and another procedure for a participant to make a bid Each Participant program provides following procedures
To allow auctioneer to call a participant to announce the onset of the session To allow auctioneer to inform a participant of a new highest bid To allow auctioneer to announce the end of session

Distributed Computing Paradigms

Distributed Object Paradigms
Provides the Object Oriented abstraction 1. Remote Method Invocation (RMI)
Object Oriented extension to the RPC Auction system is same as RPC except that object methods replaces the procedures Process issues the request to ORB which directs the request to the appropriate object ORB acts as middleware Allows interaction among objects written in different languages

2. Object Request Broker

Distributed Computing Paradigms

Object Requestor Object Object Request Broker

Implementation of auction system is same as RMI except that each object must register with ORB Each participant issues requests to the auctioneer object to register for the session and to make bids Example : OMGs CORBA Through the ORB, the auctioneer invokes the methods of each participant to announce the start of the session, to update the bidding status and to announce the end of the session

Distributed Computing Paradigms

3. Component-based Technologies
Microsofts COM, DCOM JAVA beans, EJBs

Distributed Computing Paradigms

Object Space Paradigm
Assumes the existence of the logical entity called object space The participants of an application converge in a common object space Provider places object as entries into object space, and requestors who subscribe to the space may access the entries Provides virtual space or meeting room among providers and requesters

Distributed Computing Paradigms

Requestor Provider

read

Requestor

read

writ e

Mutual exclusion is inherent as only one participant can access at an object at a time from space JavaSpaces is an example toolkit

Distributed Computing Paradigms

Example system
All the participants as well as service provider subscribes to a common object space Each participant deposits an object into space to register for the session and to be notified when the session starts. At the onset of the session, auctioneer deposits an object into object space containing item info. And bid history A participant wishing to place a bid retrieves the object from the space and if chooses, places the new bid in the object before returning object to the space At the end of session, auctioneer retrieves the object from the space and contacts the highest bidder.

Distributed Computing Paradigms

Mobile Agent Paradigm
Mobile agent is transportable program or object Agent is launched by an originating host, travels autonomously from host to host, access the necessary resources and performs the necessary tasks to complete the mission
Host 2

Host1 Agent Host 3

Host4

Distributed Computing Paradigms

Example System
Each participant launches the mobile agent to the auctioneer containing identity (network address) Once the session starts, auctioneer launches a mobile agent that carries the info. about participants and current highest bid The mobile agent circulates among the participants and auctioneer until session ends At the end auctioneer launches the agent to make one more round among the participants to announce the outcome

Commercial Packages : Concordia, Aglet system

Distributed Computing Paradigms

Network Service Paradigm
Each service provider registers with the directory servers on a network Process desires to get the service contacts the directory service to get the reference to the service Using reference, process accesses the service Extension to RMI except registration with global directory service, provides location transparency
Directory Service

Service Object

Service Requestor

Distributed Computing Paradigms

Javas Jini technology is based on this paradigm

Example system
Auctioneer registers itself with directory service, allowing the participants to locate it and once the session starts , to make the bid Participant provides call back methods to allow auctioneer to announce the start and end of the session and to update the status of the session

Distributed Computing Paradigms

Collaborative Application (GroupWare) Paradigm
Processes participates in collaborative session as a group Each process may contribute input to part or all of the group Either multicasting is used to send data to part or group or they may use virtual sketchpads or whiteboards which allows each participant to read or write data

Distributed Computing Paradigms

Message Based and White-board based groupware
message

message

message

Java multicast API and Java Shared Data ToolKit (JSDT) are message-based SMART board and Microsofts Netmeeting are based on white-board meeting

Distributed Computing Paradigms

Example system (Message-based)
Auctioneer initiates group, joined by all participants Auctioneer announces the start of session by multicast During session, each bid is multicast to all participants Finally, auctioneer terminates session by multicasting a message announcing the outcome

Example system (Whiteboard-based)

Auctioneer starts bidding by writing announcement to whiteboard Participant may write their bids on board Auctioneer write the outcome on the board

TradeTrade-offs
Level of abstraction Vs. Overhead
Higher the abstraction, higher the overhead

Scalability
Complexity increases as no. of participants increases High level paradigms, complexity is handled by system

Cross Platform Support

Java RMI, JavaSpaces runs on only JAVA COM, DCOM is only for Microsoft CORBA is cross platform

Reference
Distributed Computing : Principals and Applications - M.L. Liu - Pearson Addison-Wesley