AWS Administration – The Definitive Guide

Table of Contents

AWS Administration – The Definitive Guide

Credits

About the Author

About the Reviewer

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Instant updates on new Packt books

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Introducing Amazon Web Services

What is cloud computing?

Cloud computing features and benefits

Cloud computing use cases

Introducing Amazon Web Services

AWS architecture and components

Regions and availability zones

AWS platform overview

Compute

Storage

Databases

Networking

Distributed computing and analytics

Content distribution and delivery

Workflow and messaging

Monitoring

Identity and access management

Getting started with AWS

Introducing the AWS Management Console

Getting started with AWS CLI

Plan of attack!

Summary

2. Security and Access Management

Security and clouds

Is AWS really secure

Shared responsibility model

Identity and Access Management

Business use case scenario

Getting started with the IAM Console

Creating users and groups

Understanding permissions and policies

Creating and assigning policies

Managing access and security using the AWS CLI

Planning your next steps

Recommendations and best practices

Summary

3. Images and Instances

Introducing EC2!

EC2 use cases

Introducing images and instances

Understanding images

Amazon Linux AMI

Understanding instances

EC2 instance pricing options

On-demand instances

Reserved instances

Spot instances

Working with instances

Stage 1 – choose AMI

Stage 2 – choose an instance type

Stage 3 – configure instance details

Stage 4 – add storage

Stage 5 – tag instances

Stage 6 – configure security groups

Stage 7 – review instance launch

Connecting to your instance

Configuring your instances

Launching instances using the AWS CLI

Stage 1 – create a key pair

Stage 2 – create a security group

Stage 3 – add rules to your security group

Stage 4 – launch the instance

Cleaning up!

Planning your next steps

Recommendations and best practices

Summary

4. Security, Storage, Networking, and Lots More!

An overview of security groups

Understanding EC2 networking

Determining your instances IP addresses

Working with Elastic IP addresses

Create an Elastic IP address

Allocating Elastic IP addresses

Disassociating and releasing an Elastic IP address

Understanding EBS volumes

EBS volume types

Getting started with EBS Volumes

Creating EBS volumes

Attaching EBS volumes

Accessing volumes from an instance

Detaching EBS volumes

Managing EBS volumes using the AWS CLI

Backing up volumes using EBS snapshots

Planning your next steps

Recommendations and best practices

Summary

5. Building Your Own Private Clouds Using Amazon VPC

An overview of Amazon VPC

VPC concepts and terminologies

Subnets

Security groups and network ACLs

Routing tables

VPC endpoints

Internet Gateways

NAT instances

DNS and DHCP Option Sets

VPC limits and costs

Working with VPCs

VPC deployment scenarios

Getting started with the VPC wizard

Viewing VPCs

Listing out subnets

Working with route tables

Listing Internet Gateways

Working with security groups and Network ACLs

Launching instances in your VPC

Creating the web servers

Creating the database servers

Planning next steps

Best practices and recommendations

Summary

6. Monitoring Your AWS Infrastructure

An overview of Amazon CloudWatch

Concepts and terminologies

Metrics

Namespaces

Dimensions

Time stamps and periods

Units and statistics

Alarms

CloudWatch limits and costs

Getting started with CloudWatch

Monitoring your account's estimate charges using CloudWatch

Monitoring your instance's CPU Utilization using CloudWatch

Monitoring your instance's memory and disk utilization using CloudWatch Scripts

Creating CloudWatch access roles

Installing the CloudWatch monitoring scripts

Viewing the custom metrics from CloudWatch

Monitoring logs using CloudWatch Logs

CloudWatch Log concepts and terminologies

Getting Started with CloudWatch Logs

Viewing the logs

Creating metric filters and alarms

Planning your next steps

Recommendations and best practices

Summary

7. Manage Your Applications with Auto Scaling and Elastic Load Balancing

An overview of Auto Scaling

Auto scaling components

Auto scaling groups

Launch configurations

Scaling plans

Introducing the Elastic Load Balancer

Creating your first Elastic Load Balancer

Step 1 – Defining the Load Balancer

Step 2 – Assign security groups

Step 3 – configure security settings

Step 4 – Configure Health Check

Step 5 – Add EC2 instances

Step 6 – Add tags

Step 7 – Review and Create

Getting started with Auto Scaling

Creating the Launch Configuration

Step 1 – Choose AMI

Step 2 – Choose Instance type

Step 3 – Configure details

Step 4 – Add storage

Step 5 – Configure Security Group

Step 6 – Review

Creating the Auto Scaling Group

Step 1 – Configure Auto Scaling group details

Step 2 – Configure scaling policies

Step 3 – Configure notifications

Step 4 – Configure tags

Step 5 – Review

Verifying and testing Auto Scaling

Suspend, resume and delete Auto Scaling

Planning your next steps

Recommendations and best practices

Summary

8. Database-as-a-Service Using Amazon RDS

An overview of Amazon RDS

RDS instance types

Multi-AZ deployments and Read Replicas

Working with Amazon RDS

Getting started with MySQL on Amazon RDS

Creating a MySQL DB instance

Step 1 – Select Engine

Step 2 – Production?

Step 3: Specify DB Details

Step 4: Configure Advanced Settings

Connecting remotely to your DB instance

Testing your database

Modifying your DB instances

Backing up DB instances

Creating Read Replicas and promoting them

Logging and monitoring your DB instance

Cleaning up your DB instances

Planning your next steps

Recommendations and best practices

Summary

9. Working with Simple Storage Service

Introducing Amazon S3

Getting started with S3

Creating buckets

Uploading your first object to a bucket

Viewing uploaded objects

Accessing buckets and objects using S3CMD

Managing an object's and bucket's permissions

Using buckets to host your websites

S3 events and notification

Bucket versioning and lifecycle management

Cross-Region Replication

Planning your next steps

Recommendations and best practices

Summary

10. Extended AWS Services for Your Applications

Introducing Amazon Route53

Working with Route53

Creating hosted zones

Getting started with traffic flow

Configuring health checks

Content delivery using Amazon CloudFront

Getting started with distributions

CloudFront recommendations and best practices

What's new in AWS?

Elastic Container Service

Elastic File System

Database migration made easy with Database Migration Service

Go serverless with AWS Lambda

Resources, recommendations, and best practices

Summary

Index

AWS Administration – The Definitive Guide

AWS Administration – The Definitive Guide

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: February 2016

Production reference: 1080216

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78217-375-5

www.packtpub.com

Credits

Author

Yohan Wadia

Reviewer

Paul Deng

Commissioning Editor

Kunal Parikh

Acquisition Editor

Rahul Nair

Content Development Editor

Anish Dhurat

Technical Editor

Pranjali Mistry

Copy Editor

Charlotte Carneiro

Project Coordinator

Bijal Patel

Proofreader

Safis Editing

Indexer

Monica Ajmera Mehta

Production Coordinator

Nilesh Mohite

Cover Work

Nilesh Mohite

About the Author

Yohan Wadia is a client-focused virtualization and cloud expert with 6 years of experience in the IT industry.

He has been involved in conceptualizing, designing, and implementing large-scale solutions for a variety of enterprise customers based on VMware vCloud, Amazon Web Services, and Eucalyptus Private Cloud.

His community-focused involvement also enables him to share his passion for virtualization and cloud technologies with peers through social media engagements, public speaking at industry events, and through his personal blog—yoyoclouds.com

He is currently working with an IT services and consultancy company as a Cloud Solutions Lead and is involved in designing and building enterprise-level cloud solutions for internal as well as external customers. He is also a VMware Certified Professional and a vExpert (2012 and 2013).

I wish to dedicate this book to both my loving parents, Ma and Paa. Thank you for all your love, support, encouragement, and patience. I would also like to thank the entire Packt Publishing team, especially Ruchita Bhansali, Athira Laji, and Gaurav Sharma, for their excellent guidance and support.

And finally, a special thanks to one of my favorite bunch of people: the amazing team of developers, support staff, and engineers who work at AWS for such an AWSome cloud platform!

Not all those who wander are lost.

- J. R. R. Tolkien

About the Reviewer

Paul Deng is a senior software engineer with over 8 years of experience in end-to-end IoT app design and development, including embedded devices, large-scale machine learning, cloud, and web apps.

Paul holds software algorithm patents and was a finalist of Shell Australian Innovation Challenge 2011. He has authored several publications on IoT and cloud.

He lives in Melbourne, Australia, with his wife Cindy and son Leon. Visit his website at http://dengpeng.de to see what he is currently exploring and to learn more about him.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Instant updates on new Packt books

Get notified! Find out when new books are published by following @PacktEnterprise on Twitter or the Packt Enterprise Facebook page.

Preface

Cloud computing has definitely matured and evolved a lot ever since its conception. Practically all major industries and top fortune 500 companies today run their application workloads on clouds to reap all sorts of benefits, ranging from reduced costs, better availability of their applications, and easier manageability to on-demand scalability, and much more! At the forefront of this cloud innovation is a market leader like no other: Amazon Web Services (AWS).

AWS provides a ton of easy-to-use products and services that you can leverage to build, host, deploy, and manage your applications on the cloud. It also provides a variety of ways to interact with these services, such as SDKs, APIs, CLIs, and even a web-based management console.

This book is a one stop shop where you can find all there is to getting started with the core AWS services, which include EC2, S3, RDS, VPCs, and a whole lot more! If you are a sysadmin or an architect or someone who just wants to learn and explore various aspects of administering AWS services, then this book is the right choice for you! Each chapter of this book is designed to help you understand the individual services' concepts as well as gain hands-on experience by practicing simple and easy to follow steps. The chapters also highlight some key best practices and recommendations that you ought to keep in mind when working with AWS.

What this book covers

Chapter 1, Introducing Amazon Web Services, covers the introductory concepts and general benefits of cloud computing along with an overview of Amazon Web Services and its overall platform. The chapter also walks you through your first AWS signup process, and finally ends with the configuration of the AWS CLI.

Chapter 2, Security and Access Management, discusses the overall importance of security and how you can achieve it using an AWS core service known as Identity and Access Management (IAM). The chapter walks you through the steps required to create and administer AWS users, groups, as well as how to create and assign permissions and policies to them.

Chapter 3, Images and Instances, provides hands-on knowledge about EC2 instances and images, and how you can create and manage them using both the AWS Management Console as well as the AWS CLI.

Chapter 4, Security, Storage, Networking and Lots More!, discusses some of the key aspects that you can leverage to provide added security for your applications and instances. The chapter also provides an in-depth overview of EC2 instance storage as well as networking options followed by some recommendations and best practices.

Chapter 5, Building Your Own Private Clouds Using Amazon VPC, introduces you to the concept and benefits provided by AWS Virtual Private Cloud (VPC) service. The chapter also provides an in-depth look at various VPC deployment strategies and how you can best leverage them for your own environments.

Chapter 6, Monitoring Your AWS Infrastructure, covers AWS's primary monitoring service, called as Amazon CloudWatch. In this chapter, you will learn how to effectively create and manage alerts, loggings, and notifications for your EC2 instances, as well as your AWS environment.

Chapter 7, Manage Your Applications with Auto Scaling and Elastic Load Balancing, discusses some of the key AWS services that you should leverage to create a dynamically scalable and highly available web application.

Chapter 8, Database-as-a-Service Using Amazon RDS, provides an in-depth look at how you can effectively design, create, manage, and monitor your RDS instances on AWS.

Chapter 9, Working with Simple Storage Service, provides practical knowledge and design considerations that you should keep in mind when working with Amazon's infinitely scalable and durable object storage known as Amazon S3.

Chapter 10, Extended AWS Services for Your Application, provides a brief overview of add-on AWS services that you can leverage for enhancing your applications' performance and availability.

What you need for this book

To start using this book, you will need the following set of software installed on your local desktop:

An SSH client such as Putty, a key generator such as PuttyGen, and a file transferring tool such as WinSCP

Any modern web browser, preferably Mozilla Firefox

Who this book is for

This book is intended for any and all IT professionals who wish to learn and implement AWS for their own environments and application hosting. Although no prior experience or knowledge is required, it will be beneficial for you to have basic Linux knowledge as well as some understanding of networking concepts and server virtualization.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: We can include other contexts through the use of the include directive.

A block of code is set as follows:

{

Id: Policy1448937262025,

Version: 2012-10-17,

Statement: [

  {

Sid: Stmt1448937260611,

Effect: Allow,

Principal: *,

  }

}

Any command-line input or output is written as follows:

CREATE TABLE doge ( idint(11) NOT NULL auto_increment, namevarchar(255), description text,

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: Next, select the Launch DB Instance button to bring up the DB Launch Wizard:

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <feedback@packtpub.com>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <copyright@packtpub.com> with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at <questions@packtpub.com>, and we will do our best to address the problem.

Chapter 1. Introducing Amazon Web Services

Being in the IT industry, we all have gone through the long and tedious process of procuring new infrastructure for our data centers at one time or another. Let's be honest, it is not a task for the faint-hearted At a minimum, a brand new server can take weeks if not months to get delivered from its date of ordering, and this is too heavily dependent on a lot of external factors which, most of the time, are not in our control. Even if the new server comes in on time, there is the additional burden of prepping the rack, clearing space for the new resources, cabling, cooling, mounting, installation of software, configuration, and the list just keeps on going on for another mile. Putting the server and storage on one side, the same can also apply for networking, applications, software, and a whole lot of other things. But what if all this was to change? What if tomorrow, you could simply jump start your business or scale your application to thousands of servers, all with the simple click of a button? Seems farfetched, doesn't it, but in reality, this is possible today with a little help from something called as cloud computing.

Cloud computing has definitely evolved a lot over the years, and today it has become almost a mainstream part of our lives. Everything from storing large amounts of data, to having burst compute capacity at your fingertips and having enterprise software applications available on demand any time anywhere are just some of the key benefits that clouds provide today. At the helm of this new way of computing is Amazon Web Services (AWS).

In this chapter, we are going to look at some of the key features and benefits provided by cloud computing in general along with few interesting enterprise use cases. Later on, you will learn a bit more about Amazon Web Services and its core components, and finally have a look at how to sign up and get started with AWS.

What is cloud computing?

Cloud computing has become one of the most discussed topics over the last few years, but what does it actually mean? Why is it important for you and your business? Let's take a quick look at what cloud computing actually is and how you as an end user can benefit from it.

NIST defines cloud computing as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (for example, networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. Was that a bit confusing? Let's break it down to understand it a little bit better:

On demand: Contrary to traditional IT data centers where requesting and obtaining resources for your applications used to take weeks, the same resources can be made available on an on-demand basis in the cloud, without requiring any human interventions at all.

Network access: One of the key features of any cloud is that all of its resources can be accessed over the Internet or a network in general. This makes it really easy for the end user to access and leverage cloud from heterogeneous devices such as laptops, workstations, mobile phones, and so on.

Shared pool: We must have all experienced the silo-based infrastructure setup in our traditional IT data centers. Silos of resources are often created by individual departments where each application gets overcommitted resources, which are more than often wasted. In case of the cloud, IT resources such as compute, storage, and network are pooled and abstracted together from the end user. This pooling enables the cloud to dynamically provide resources to its tenants as and when required without the end user having any knowledge of where their application will actually reside in a cluster or a rack.

Rapid provisioning: Rapid provisioning or elasticity as it is referred to nowadays, is the ability of a cloud to scale its resources either horizontally (scale out) or vertically (scale up) on an automated basis. This elasticity provides end users with a lot of flexibility and control over their resources and how they get consumed.

Cloud computing features and benefits

Okay, so now we know what cloud computing really means, but why should you as an end user use it? How are you and your organization going to benefit from it? Let's look at some of the key benefits and features cloud computing has to offer:

Lower costs: Cloud computing does not have any upfront costs. Capital expenditure is virtually zero as the entire necessary infrastructure and resources are already made available and ready for use by the Cloud provider. All the end user has to do is consume these resources and pay only for what they use.

This also reduces the overall maintenance costs as well. Since the organization now has less hardware to manage, it doesn't necessarily have to bother about its server, network, and storage capacity or have a full time staff to manage them.

Self-service: Cloud computing provides a simple, centralized, self-service-based model that end users can use to interact with the cloud platform. The interaction can be in terms of performing simple tasks, such as spinning up new compute resources, adding more storage resources at runtime, or more complex ones, such as scheduling resources and so on. This ensures that the end user can leverage and consume any of the cloud's resources on an on-demand basis.

Faster time to markets: Unlike traditional IT, compute resources can be brought up in a cloud in a matter of minutes as compared to the weeks it used to take. This provides the end users with the ability to deploy new applications much faster, thus decreasing overall time to markets and reducing management overheads and costs.

Scale as required: The best part of using the cloud is that your backend resources can grow as your application grows. This means that you are never lagging behind with your application's needs and demands, you are always scaling with your application's needs.

Cloud computing use cases

With such features and benefits it is easy to see why there has been such a boom in the overall adoption and utilization of the cloud. Let's take a quick look at some interesting real-world use cases where your organization can leverage clouds:

Website hosting: Perhaps the most common of the use cases, you as an end user can leverage cloud to build and host your websites with relative ease. The cloud enables your website to scale up and down dynamically as per its demands.

Storage and sharing: The cloud offers virtually unlimited storage capacity that can used to store and share anything, from documents, media, files, and so on. Dropbox and Google Drive are classic examples of cloud being used as a storage and sharing medium.

Disaster recovery (DR): This is a more upcoming use case with clouds as more and more companies are now realizing that it is way easier and cost efficient to host a disaster recovery environment on the cloud, rather than hosting and managing a DR site of its own. Organizations can spin up failover environments on the cloud in a matter of minutes, test the failover and then shut down the entire stack. This helps save on the costs and also reduces overall management overheads and failover time.

Dev/test: Dev and test are way easier to set up and run on clouds as the