CloudBridge 1.

0 Installation Guide

CloudBridge Installation Guide

Version 1.0 release

Document revised May 22, 2011

2010, 2011 Cloud.com, Inc. All rights reserved.

CloudBridge 1.0 Installation Guide

2010, 2011 Cloud.com, Inc. All rights reserved. Specifications are subject to change without notice. The Cloud.com logo, Cloud.com, CloudStack, are trademarks or registered trademarks of Cloud.com, Inc. All other brands or products are trademarks or registered trademarks of their respective holders.
2010, 2011 Cloud.com, Inc. All rights reserved. 2

CloudBridge 1.0 Installation Guide

Contents
1 2 3 4 Overview ............................................................................................................................................................................................ 4 Prerequisites ...................................................................................................................................................................................... 5 Deployment Architecture .................................................................................................................................................................. 6 CloudBridge Server Installation ......................................................................................................................................................... 7 4.1 4.2 4.3 4.4 4.4.1 5 Operating System and OS Preparation ..................................................................................................................................... 7 First CloudBridge Server Install ................................................................................................................................................ 7 Additional CloudBridge Server Installs (Optional) .................................................................................................................... 8 Database Replication (Optional)............................................................................................................................................. 10 Failover ............................................................................................................................................................................... 11

Using the CloudBridge ..................................................................................................................................................................... 12 5.1 5.2 User Registration .................................................................................................................................................................... 12 Using the ec2-api-tools ........................................................................................................................................................... 12

6 7

Troubleshooting ............................................................................................................................................................................... 13 Appendix A -- Available Commands ................................................................................................................................................. 14

2010, 2011 Cloud.com, Inc. All rights reserved.

CloudBridge 1.0 Installation Guide

Overview

The CloudBridge is a server process that runs as an adjunct to the CloudStack. The CloudBridge provides an Amazon EC2 compatible API via both SOAP and REST web services. The EC2 API is translated to native CloudStack API calls by the CloudBridge. Clients can continue using existing EC2-compatible tools with the CloudBridge. The CloudBridge requires a RHEL5/CentOS server that is separate from the CloudStack Management Server. The CloudBridge connects to the CloudStack Management Server to implement the user requests.

2010, 2011 Cloud.com, Inc. All rights reserved.

CloudBridge 1.0 Installation Guide

Prerequisites

The CloudBridge has the following hardware and software requirements. Description CloudBridge Server Hosts the CloudBridge software. Minimum Requirements 64-bit x86 CPU (more cores results in better performance) 2 GB of memory 80 GB of local disk At least 1 NIC RHEL/CentOS 5.4+ 64-bit Statically allocated IP address Fully qualified domain name as returned by the hostname command

EC2 tools v. 1.3.6230

Client interface to the Amazon EC2 service.

CloudBridge works with version 1.3.6230 of the EC2 Tools. Download the correct version here: http://s3.amazonaws.com/ec2-downloads/ec2-api-tools1.3-62308.zip

2010, 2011 Cloud.com, Inc. All rights reserved.

CloudBridge 1.0 Installation Guide

Deployment Architecture

The architecture used in a deployment will vary depending on the size and purpose of the deployment. All CloudBridge deployments include a MySQL server to store credential mappings. The CloudBridge API client sets up a mapping from a user's EC2 certificate to their CloudStack API keys. This mapping is stored in the MySQL database. All CloudBridge deployments include at least one CloudBridge server. The CloudBridge server runs in the Tomcat container. Trial deployments may use a single CloudBridge server. Production deployments generally use two CloudBridge servers that are implemented behind a load balancer. The load balancer need not establish sticky sessions for the CloudBrige access. The CloudBridge server listens on port 8090 by default. You may map port 80 of a public IP address to port 8090 of your CloudBridge servers. The CloudBridge can access the CloudStack Management Server via either a public or private IP address. If you have multiple CloudStack Management Servers you should configure a load balanced IP address for the CloudStack Management Server to be used by CloudBridge. The CloudBridge will by default use port 8080 to communicate with the CloudStack Management Server.

CloudBridge Load Balanced IP

CloudBridge Server

CloudBridge Server

CloudStack Management Server Load Balanced IP

2010, 2011 Cloud.com, Inc. All rights reserved.

CloudBridge 1.0 Installation Guide

CloudBridge Server Installation

The Cloud.com CloudBridge Server download includes everything you need to get started, except MySQL. This includes the Cloud.com software as well as dependencies. This section describes installing one or more CloudBridge Servers with one instance of MySQL, which may be on a different node from the CloudBridge Servers. The procedure for the installation is: 1. 2. 3. 4. Prepare the operating system for all CloudBridge servers. Install the first CloudBridge Server. Install MySQL. (optional) Install additional CloudBridge Servers to create a farm for high availability.

To simplify the installation procedure this document defines two separate installation procedures: one for installing a single CloudBridge Server and one for installing multiple CloudBridge Servers in a load balanced pool. This document assumes that, in the case of multiple CloudBridge Servers, MySQL will be installed on a separate node from the CloudBridge Servers.

4.1 Operating System and OS Preparation

The CloudBridge Server requires RHEL/CentOS 5.4 64 bit or later. You can download CentOS 64-bit via the following link: http://isoredirect.centos.org/centos/5/isos/x86_64/. The OS must be prepared to host the CloudBridge Server using the following steps. Important: These steps should be done on all CloudBridge Servers. 1. 2. 3. 4. Edit the /etc/hosts file to make sure that every CloudBridge Server has a fully-qualified host name that resolves to an IP address. Alternatively, you can do this through DNS. Log in to your OS as root. All the following commands should be run as root. Ensure that the SELINUX variable in /etc/selinux/config is set to permissive. This ensures that MySQL and the CloudBridge Server can run properly on system reboot. Run the following command. # setenforce permissive 5. Make sure that the CloudBridge Server can reach the Internet. # ping www.google.com 6. Open up port 8090 on the CloudBridge server. # iptables I INPUT p tcp --dport 8090 j ACCEPT 7. Edit the /etc/sysconfig/iptables file and add the following lines at the beginning of the INPUT chain. This will keep 8090 open on host reboot. -A INPUT p tcp --dport 8090 j ACCEPT

4.2 First CloudBridge Server Install

This section describes the procedure for performing the installation of a CloudBridge server where the CloudBridge Server and MySQL are on a single, shared OS instance. If you have multiple CloudBridge Servers you should follow this step then add additional CloudBridge servers as discussed in section 4.3.
2010, 2011 Cloud.com, Inc. All rights reserved. 7

CloudBridge 1.0 Installation Guide These steps assume that the preparation steps at the beginning of section 4 have been performed. 1. Install prerequisite software. This includes MySQL, Java, and Tomcat. # # # # # # yum install mysql mysql-server service mysqld start yum install java cd /etc/yum.repos.d wget 'http://www.jpackage.org/jpackage50.repo' yum install tomcat6 tomcat6-webapps tomcat6-admin-webapps

2.

Install the CloudBridge package. You should have a file in the form of cloud-bridge-xxx.rpm: # rpm -ivvh cloud-bridge-xxx.rpm

3.

Set up the bridge's configuration. In this step you will create a mapping from common EC2 service offerings to CloudStack service offerings. The CloudStack service offerings are referred to by their Service Offering IDs. These IDs are available in the CloudStack Admin UI. # cloud-setup-bridge Welcome to the CloudBridge setup. Enter suitable values or press enter for default. Management server hostname or IP [127.0.0.1]: 92.52.146.124 Management server port [8080]: 80 Service offering ID for m1.small [1]: Service offering ID for m1.large [2]: Service offering ID for m1.xlarge [4]: 2 Service offering ID for c1.medium [3]: 2 Service offering ID for c1.xlarge [3]: 2 Service offering ID for m2.xlarge [3]: 2 Service offering ID for m2.2xlarge [3]: 2 Service offering ID for m2.4xlarge [3]: 2 Service offering ID for cc1.4xlarge [3]: 2 Values saved. Restart the cloud-bridge service for the changes to become active.

4.

Setup the CloudBridge database schema and start the CloudBridge service. # cloud-setup-bridge-db # service cloud-bridge start

This completes the single node install for the CloudBridge Server and database.

4.3 Additional CloudBridge Server Installs (Optional)

This section describes adding an additional CloudBridge server to an existing set of one or more CloudBridge servers and MySQL server.
2010, 2011 Cloud.com, Inc. All rights reserved. 8

CloudBridge 1.0 Installation Guide Important: These steps assume that the preparation steps at the beginning of section 4 have been done. The procedure to install a second CloudBridge server is: 1. 2. 3. 4. Prepare the server as at the beginning of section 4. Install the CloudBridge RPM Run the setup script for the CloudBridge Edit configuration to use the existing MySQL server

These steps are as follows. 1. Install prerequisite software. This includes MySQL, Java, and Tomcat. # # # # yum install java cd /etc/yum.repos.d wget 'http://www.jpackage.org/jpackage50.repo' yum install tomcat6 tomcat6-webapps tomcat6-admin-webapps

2.

Install the CloudBridge package. You should have a file in the form of cloud-bridge-xxx.rpm". # rpm -ivvh cloud-bridge-xxx.rpm

3.

Set up the bridge's configuration. For these steps you should provide the same answers as were given when the first CloudBridge server was installed. # cloud-setup-bridge Welcome to the CloudBridge setup. Enter suitable values or press enter for default. Management server hostname or IP [127.0.0.1]: 92.52.146.124 Management server port [8080]: 80 Service offering ID for m1.small [1]: Service offering ID for m1.large [2]: Service offering ID for m1.xlarge [4]: 2 Service offering ID for c1.medium [3]: 2 Service offering ID for c1.xlarge [3]: 2 Service offering ID for m2.xlarge [3]: 2 Service offering ID for m2.2xlarge [3]: 2 Service offering ID for m2.4xlarge [3]: 2 Service offering ID for cc1.4xlarge [3]: 2 Values saved. Restart the cloud-bridge service for the changes to become active.

4.

Edit the configuration for the second server to use the already-installed MySQL instance on the first CloudBridge Server. In the file /usr/share/cloud/bridge/conf/ec2-service.properties set dbHost to the hostname of the first CloudBridge server. Then edit /usr/share/cloud/bridge/conf/hibernate.cfg.xml and set hibernate.connection.url to the hostname of the first CloudBridge server.
2010, 2011 Cloud.com, Inc. All rights reserved. 9

CloudBridge 1.0 Installation Guide # cd /usr/share/cloud/bridge/conf # vi ec2-service.properties # vi hibernate.cfg.xml Finally, restart the cloud-bridge service. # service cloud-bridge restart Your additional CloudBridge server is now available for service.

4.4 Database Replication (Optional)

CloudBridge supports database replication from one MySQL node to another. This is achieved using standard MySQL replication. You may want to do this as insurance against MySQL server or storage loss. MySQL replication is implemented using a master/slave model. The master is the node that the CloudBridge Servers are configured to use. The slave is a standby node that receives all write operations from the master and applies them to a local, redundant copy of the database. The following steps are a guide to implementing MySQL replication. Important: These steps assume that this is a fresh install with no data in the master. Important: Creating a replica is not a backup solution. You should develop a backup procedure for the MySQL data that is distinct from replication. 1. Edit my.cnf on the master and add the following in the [mysqld] section below datadir. log_bin=mysql-bin server_id=1 For server_id a common practice is to set it to the last octet of the servers IP address. It must be unique with respect to other servers. Restart the MySQL service: # service mysqld restart 2. Create a replication account on the master and give it privileges. We will use the cloud-repl user with the password password. This assumes that master and slave run on the 172.16.1.0/24 network. # mysql -u root mysql> create user 'cloud-repl'@'172.16.1.%' identified by 'password'; mysql> grant replication slave on *.* TO 'cloud-repl'@'172.16.1.%'; mysql> flush privileges; mysql> flush tables with read lock; 3. 4. 5. Leave the current MySQL session running. In a new shell start a second MySQL session. Retrieve the current position of the database. # mysql -u root mysql> show master status; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql-bin.000001 | 412 | | |
2010, 2011 Cloud.com, Inc. All rights reserved. 10

CloudBridge 1.0 Installation Guide +------------------+----------+--------------+------------------+ 6. 7. 8. Note the file and the position that are returned by your instance. Exit from this session. Complete the master setup. Returning to your first session on the master, release the locks and exit MySQL. mysql> unlock tables; 9. Install and configure the slave. On the slave server, run the following commands. # yum install mysql-server # chkconfig mysqld on 10. Edit my.cnf and add the following lines in the [mysqld] section below datadir. server_id=2 innodb_rollback_on_timeout=1 innodb_lock_wait_timeout=600 11. Restart MySQL. # service mysqld restart 12. Instruct the slave to connect to and replicate from the master. Replace the IP address, password, log file, and position with the values you have used in the previous steps. mysql> -> -> -> -> -> change master to master_host='172.16.1.217', master_user='cloud-repl', master_password='password', master_log_file='mysql-bin.000001', master_log_pos=412;

13. Then start replication on the slave. mysql> start slave; 14. Optionally, open port 3306 on the slave as was done on the master earlier. Important: This is not required for replication to work. But if you choose not to do this you will need to do it when failover to the replica occurs.

4.4.1 Failover
This will provide for a replicated database that can be used to implement manual failover for the CloudBridge Servers. CloudStack failover from one MySQL instance to another is performed by the administrator. In the event of a database failure you should: 1. 2. 3. 4. 5. Stop the CloudBridge Servers (via service cloud-bridge stop). Change the replicas configuration to be a master and restart it. Ensure that the replicas port 3306 is open to the CloudBridge Servers Make a change so that the CloudBridge uses the new database. Restart the CloudBridge Servers (via service cloud-bridge restart)
2010, 2011 Cloud.com, Inc. All rights reserved. 11

CloudBridge 1.0 Installation Guide

Using the CloudBridge

In general the CloudBridge users need not be aware that they are using a CloudBridge server. They need only execute EC2 style API calls to the CloudBridge and it will translate the calls to the CloudStack's native API. However, each user must perform a registration step with the CloudBridge that is unique to it. This is a one-time procedure and is discussed in section 5.1. After this is done the users have to set up their environment and/or tools appropriately to use the CloudBridge endpoint. This is discussed in the case of the EC2 tools in section 5.2.

5.1 User Registration

After the CloudBridge software is installed each CloudBridge user must perform a one-time registration process to create the certificate to API key mapping discussed earlier. This is a one-time procedure. It should be done once for the CloudBridge install regardless of the number of CloudBridge servers that have been installed. The steps to do this are as follows. 1. Generate a private key and a self-signed X.509 certifcate with openssl. $ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /path/to/private_key.pem -out /path/to/cert.pem 2. 3. Obtain your API key and Secret key from your CloudStack cloud administrator. Register the mapping from X.509 certificate to the API/Secret keys in the CloudBridge. The users will execute this command so you should be sure to provide them with the publicly available DNS name or IP address for the URL argument that is the CloudBridge server. $ cloud-bridge-register --apikey=<Users Cloudstack API key> --secretkey=<Users CloudStack Secret key> --cert=</path/to/cert.pem> --url=http://cloud-bridgeserver:8090/bridge Now the user is established in the CloudBridge and may make EC2-style calls to it.

5.2 Using the ec2-api-tools

After registering the certificate and keys with CloudBridge, perform these setup steps: 1. 2. Be sure you have the right version of EC2 Tools (http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-62308.zip). Set up the environment variables that will direct the tools to the CloudBridge server. The steps to do so are as follows. As a best practice, you may wish to place these in a script that may be sourced before CloudBridge use. $ $ $ $ $ export export export export export EC2_ACCESS_KEY=<CloudStack API key> EC2_SECRET_KEY=<CloudStack Secret key> EC2_CERT=/path/to/cert.pem EC2_PRIVATE_KEY=/path/to/private_key.pem EC2_URL=http://cloud-bridge-hostname:8090/bridge

Now the EC2 commands may be used.

2010, 2011 Cloud.com, Inc. All rights reserved.

12

CloudBridge 1.0 Installation Guide

Troubleshooting

The CloudBridge logs to /usr/share/cloud/bridge/logs/.

2010, 2011 Cloud.com, Inc. All rights reserved.

13

CloudBridge 1.0 Installation Guide

Appendix A -- Available Commands

The following EC2 commands are available with the CloudBridge. Addresses AllocateAddress AssociateAddress DescribeAddresses DisassociateAddress ReleaseAddress Availability Zones DescribeAvailabilityZones Images CreateImage DeregisterImage DescribeImages RegisterImage Image Attributes DescribeImageAttribute ModifyImageAttribute ResetImageAttribute Instances DescribeInstances RunInstances (Requires parameters availability zone, min count, and max count. Refer to the desired template by template ID.) RebootInstances StartInstances StopInstances TerminateInstances Instance Attributes DescribeInstanceAttribute Keypairs CreateKeyPair DeleteKeyPair DescribeKeyPairs ImportKeyPair Passwords GetPasswordData Security Groups AuthorizeSecurityGroupIngress CreateSecurityGroup DeleteSecurityGroup DescribeSecurityGroups RevokeSecurityGroupIngress

2010, 2011 Cloud.com, Inc. All rights reserved.

14

CloudBridge 1.0 Installation Guide Snapshots CreateSnapshot DeleteSnapshot DescribeSnapshots Volumes AttachVolume CreateVolume DeleteVolume DescribeVolumes DetachVolume

2010, 2011 Cloud.com, Inc. All rights reserved.

15