Cloud Computing - Shared Resources - Ubuntu 15.04 Server with systemd: Administration and Reference (2015)

Ubuntu 15.04 Server with systemd: Administration and Reference (2015)

Part III. Shared Resources

Chapter 13. Cloud Computing

Ubuntu features fully integrated support for cloud computing. Ubuntu provides private and public cloud support. The public cloud accesses the Amazon EC2 cloud system, and the private cloud sets up your own cloud computing service with the Ubuntu Enterprise Cloud software. Both use EC2 (Elastic Computing), which is the standard for cloud computing. Cloud support is still very much a work in progress. An overview of Ubuntu cloud computing with links is located at:

http://www.ubuntu.com/cloud/

You will need to use a Web browser to set up access and manage your cloud. Use either a command line browser like elinks or lynx, or, if you have installed the ubuntu desktop or basic GNOME interface, you can use Firefox or Epiphany.

Check the Amazon EC2 documentation for more details, including the User Guide and Getting Started guide at http://docs.amazonwebservices.com.

Public Cloud: Amazon EC2 Cloud

The Ubuntu 15.04 server edition provides support for access the Amazon EC2 cloud provided by Amazon Web Services (AWS). This is a commercial service that you have to sign up and pay for. Once you have access, you can then access an Amazon Machine Image (AMI) for an Ubuntu server and set up applications to run from the Amazon EC2 cloud. See the Ubuntu public cloud page for an overview.

http://www.ubuntu.com/cloud/

You can find out more about Amazon EC2 cloud at:

http://aws.amazon.com/ec2/

The EC2StartGUide shows how to set up access.

https://help.ubuntu.com/community/EC2StartersGuide

To find out what Ubuntu AMIs are available on Amazon, you can use the Ubuntu Cloud Images site at:

http://cloud-images.ubuntu.com

Cloud tools

You can install a tool to allow you to start and stop instances. The recommended tools are the Eucalyptus tools in the euca2ools package and the Amazon tools in the ec2-api-tools package. The Eucalyptus tool is an open source tool supported by Ubuntu.

Amazon EC2 tools

Alternatively, you can use the Amazon EC2 tools (Multiverse repository). The examples in this chapter use these tools.

First install the Amazon EC2 API package.

ec2-api-tools

There are an extensive number of EC2 tools provided by this package. A listing and explanation for these tools are located at:

http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/

Click on the API Command Line Tools Reference link on the left pane of this Web page. These explanations also apply to their Eucalyptus counterparts (euca prefix).

Eucalyptus tools

For the Eucalyptus tools, you install the euca2ools package.

euca2ools

The package installs several cloud management tools, beginning with the prefix euca. For examples check the man page for a command and also check the Eucalyptus User Guide at:

http://open.eucalyptus.com/wiki/Euca2oolsGuide

Proprietary management tools

You can also use proprietary management tools such as Canonical's Landscape, Rightscale cloud management service (http://www.rightscale.com), and the CohesiveFT Elastic cloud management support (http://www.cohesiveft.com/). Check this site for more information.

http://www.ubuntu.com/cloud/management

Setting up access

On the Amazon cloud, you can access a public Amazon Machine Image (AMI) for an Ubuntu 15.04 server system provided by Ubuntu. You will have to create an Amazon EC2 account, set up security, and then set up your cloud. Check the Ubuntu EC2 starter guide on how to set up access.

https://help.ubuntu.com/community/EC2StartersGuide

Create an account

To set up an Amazon EC2 account, you first have to have a basic Amazon account. Set one up if you do not already have one. Then sign in and set up an Amazon EC2 account at (click the Sign Up button):

http://aws.amazon.com/ec2

Set up Security:

To ensure access to the Amazon EC2 cloud, you have to make sure your security certificates and keys are installed and made available to the EC2 API tools that will manage your access to the AMI. You will create a certificate and private key on your AWS account. Click on the Account tab and choose Security Credentials. Click the X.509 Certificates tab. On this tab, click the "Create a new Certificate" link. This opens a dialog with buttons to download both a private key and certificate. Take note where you are downloading the certificate and private key. The private key file begins with the prefix pk- and the certificate file begins with cert-.

If you have set up an account already, but your Ubuntu system does not have access, you will have to generate a new certificate in order to download a private key. Sign in and click the Account tab, choosing the Security Credentials link. Click on the "X.509 Certificates" tab, and then click on the "Create a new Certificate" link. A dialog opens with buttons to download the Private key and the X.509 certificate.

Note your account ID which is listed at the bottom of the Security Credentials page.

You then set up three shell variables and export them to make them global. These are set up in your .bashrc file in your home directory. The variables hold the locations of your private key, Amazon certificate, and the JAVA OpenJDK.

EC2_PRIVATE_KEY The location and name of your Amazon EC2 private key file

EC2_CERT The location and name of your Amazon EC2 certificate file

JAVA_HOME The location of the JAVA OpenJDK software

The EC2StartGUide provides an example format.

https://help.ubuntu.com/community/EC2StartersGuide

Here is the example.

export EC2_PRIVATE_KEY=$HOME/<where your private key is>/pk-XXXXXXXXXXXXXXXXX.pem
export EC2_CERT=$HOME/<where your certificate is>/cert-XXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64/

The name of the certificate and key files can be very complex. On the command line interface, a simple way to copy the file names is to list them with the ls command and save the names in a file that you can then copy and paste from in nano.

ls *.pem > mykeyname
nano mykeyname

To access the Ubuntu Server public AMI, you also have to generate an SSH key. Use the ec2-add-keypair command to create an SSH key. Be sure to save the output to a file, named in this example myec2key.pem. You can also set up the SSH keys using the AWS console.

ec2-add-keypair myec2key > myec2key.pem

Set the file permissions to 600.

chmod 600 myec2key.pem

Authorize access through the SHH port, port 22, using the ec2-authorize command.

ec2-authorize default -p 22

You can assign the SSH key you made to the EC2_KEYPAIR variable, and then use that variable to reference the key in your ec2 commands.

export EC2_KEYPAIR=<your keypair name> # name only, not the file name

In addition, you can specify the EC2 region site for your cloud instances. These are listed on the EC2StartersGuide Web page. For example, us-east-1 indicates the Eastern US, and eu-west-1 indicates Europe. The region site URL is assigned to the EC2_URL variable.

export EC2_URL=https://ec2.<your ec2 region>.amazonaws.com

You can then edit the .bashrc file, adding the EC2 variables at the end and copying and pasting the key file names.

nano .bashrc

On a desktop interface, you can edit the .bashrc file and copy and paste using gedit (View | Show Hidden Files).

A sample of the lines you would add is listed on the EC2StartersGuide Web page and is shown here, with example key and certificate names. In this example, the keys are in the user's HOME directory ($HOME), though you may want to place them in a more secure directory. On a desktop interface you could copy and paste directly from the Web page to the .bashrc file, being edited with gedit.

export EC2_KEYPAIR=myec2key # name only, not the file name
export EC2_URL=https://ec2.us-west-1.amazonaws.comexport EC2_PRIVATE_KEY=$HOME/pk-ABCDE2MA6RCNEC7LCXEDULV7H6JBZZZZ.pem
export EC2_CERT=$HOME/cert-ABCDE2MA6RCNEC7LCXEDULV7H6JBZZZZ.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64/

Ubuntu AMI Cloud Images

The official AMI IDs for Ubuntu are listed using the Ubuntu Cloud Images site at:

http://cloud-images.ubuntu.com

The Cloud Images page requires that you have access to a Web browser.

Click on the folder with the release name you want, such as vivid for Ubuntu 15.04. This opens the page for the daily build folder. Click on the current folder for the latest link.

Clicking on Launch button opens the AWS Web console, prompting you to add an instance of the selected image.

Accessing the AMI with the ec2 commands

The EC2 Starter Guide describes the use of the Amazon EC2 tools (ec2-api-tools), but the examples also apply to the Eucalyptus EC2 tools (euca2ools). Just replace the ec2 prefix for these tools with the euca prefix, as in euca-describe-images instead of ec2-describe-images.

You will first have to find the AMI ID for the Ubuntu Server AMI. You have to reference the AMI ID to access that image. The official AMI IDs for Ubuntu are listed on the Ubuntu Cloud Images site at:

http://cloud-images.ubuntu.com

The "ec2 command" column lists the actual ec2 command you have to enter to start the instance. It shows the ec2-run-instance command with the AMI ID and the cloud region.

To list all available Ubuntu images you can use the ec2-describe-images command with the -a option, and use grep to filter the results with the ubuntu pattern. These will include images posted by third parties, not just the official Ubuntu images.

ec2-describe-images -a | grep ubuntu

To list the current official Vivid Ubuntu AMI images, use the ec2-describe-images command and pipe the results through a series of grep operations beginning with a 099720109477/ubuntu-images/ubuntu-vivid pattern. Be sure to specify the official Canonical user Amazon ID, 099720109477. Add a grep operation to show just the AMI images. This will display the original release Ubuntu images. You can add another grep operation to list just the 32 or 64 bit version.

ec2-describe-images -a | grep 099720109477/ubuntu/images/ubuntu-vivid

Add a grep operation for an instance-store, or ebs for an Elastic Block Store volume.

The images are periodically rebuilt, and have the name images-testing instead of just images. The latest (daily build) images have the name daily. You can add the image type and the date to narrow your search.

To access an Amazon Machine Image (AMI), you first run the instance using the ec2-run-instances command. You will have to specify the AMI image, the SSH keys you created (use the EC2_KEYPAIR variable), and the type (small or micro). For a 64 bit AMI specify the size. For 64 bit systems you can add the -t c1.large option.

The reservation and instance information is displayed.

Run the ec2-describe-instances command to find out your instance ID and the external host the instance is running on.

There are two lines; the second is lengthy and will wrap around. The first entry in the INSTANCE line is the instance ID, and begins with i- prefix. The AMI image follows, and then the external host name on which the instance is being run.

Once the AMI is running, you login using the ssh command, your private key, and the external host listed in the ec2-describe-instances output preceded with ubuntu@.

When finished, you can logout to return to your shell.

When you are finished, be sure to shut down your AMI instance with the ec2-terminate-instances command. Otherwise your AMI will continue to run and you will be charged for its use. For this command you use the instance ID listed in the ec2-describe-instances output.

ec2-terminate-instances <instance_id>

Use the ec2-describe-instances command to check the status of your AMI, as well as to make sure it is shut down.

Amazon AWS Management Console

If you have access to a desktop Web browser on your system, you can also use the AWS management console to manage and access your Amazon EC2 cloud. Click the EC2 tab to manage your AWS EC2 service. Tasks are listed in the Navigation panel organized by Instances, Images, EBS, and Networking & Security. You can create private keys (SSH) for access and launch instances of AMI images.

http://aws.amazon.com/console/

The easiest way to access Ubuntu Images is to use the Ubuntu Cloud Images site to first find the image you want, and then click on the Launch button, which is a link to open the AWS management console to configure an instance of that image. You are taken through the steps of creating and starting an instance, as well as creating a private key (SSH) to access the instance. Once the instance is created, you can manage it by clicking on the Instance link in the Navigation bar. You can terminate an instance, click its check box and then choose Terminate from the Instance Actions menu.

Information on creating an AMI

You can create your own AMI with the Amazon AMI tools. Install the ec2-ami-tools package.

ec2-ami-tools

To create an AMI, you use the ec2-bundle-image tool. You then use the ec2-upload-bundle tool to upload it to the Amazon EC2 cloud. The /etc/ec2/amitools directory will hold the EC2 certificate.

A listing and explanation for the EC2 AMI tools are located at:

http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/

Click on the AMI Tools Reference link on the left pane of this Web page.

Cloud-Init

The cloud-init package provides scripts for the configuration and customization of cloud instances. It applies to AMI images. It includes support for installing SSH key during instance initialization, login credentials (EC2 SSH keys), repository access, and to set your host name. The/etc/cloud-init directory holds the cloud-config.cfg configuration file specifying scripts to run, and a templates subdirectory for your hostname and Ubuntu repository configuration. See https://help.ubuntu.com/community/CloudInit for more details.

OpenStack

OpenStack is the private/public cloud software currently supported by Ubuntu, replacing Eucalyptus. You can find out more about OpenStack at:

http://www.openstack.org

Detail documentation, including the end user guide, is available at:

http://docs.openstack.org

You can use the OpenStack dashboard (horizon) to manage your cloud (Universe repository). It provides a Web interface for administering the OpenStack Nova cloud controller.

OpenStack comprises several services, including compute (Nova), storage (Cinder), and imaging (Glance), networking (Neutron), orchestration (Heat), and database support (Trove). The computer packages on Ubuntu begin with the term nova (Ubuntu main repository). OpenStack supports several kinds of hypervisors including KVM and Xen, as well as database backends such as MySQL and PostgreSQL. It also supports LDAP user databases.

Ubuntu Cloud

The Ubuntu Cloud refers to the full range of cloud software and services that Ubuntu offers. These include both Public cloud (instances of Ubuntu on public clouds such as the Amazon AWS), and Ubuntu OpenStack (OpenStack supported clouds that you can set up yourself). See:

http://www.ubuntu.com/cloud

The Ubuntu OpenStack refers to a cloud you can build yourself, using the OpenStack applications and servers provided by Ubuntu. An OpenStack cloud can be private, public, or a hybrid of both. They can be managed using Juju charms (apps).

In addition, the Ubuntu Advantage service (Cloud management) provides professional support for Ubuntu OpenStack. You can then use Landscape to manage your Ubuntu clouds. For more information see:

http://www.ubuntu.com/management

Note: The Ubuntu Enterprise Cloud and the Eucalyptus cloud software are no longer supported by Ubuntu.

Service Orchestration

To implement cloud application services, Canonical provides JuJu and Metal as a Service (MAAS). JuJu is a service orchestration toolset and MAAS provides hardware provisioning. With JuJu you can manage installation, configuration, and availability of your cloud services, with MAAS set up hardware resources for them. Juju (formerly Ensemble) is a service orchestration system, which provides the tools to configure, upgrade, and deploy cloud services (juju package, Universe repository). It allows you to monitor use and scale deployment of services. For more information see:

https://juju.ubuntu.com/

JuJu orchestrates application components such as dependencies on other software and various configuration settings, by combining them into an entity referred to as a charm. Using charms, JuJu can quickly manage a cloud service. A charm is basically a script, as set of instructions, for deploying a cloud service. Several are already available. With a JuJu charm you can easily deploy a cloud service, specifying the number of nodes to use and choosing the cloud service such as AmazonEC2 or OpenStack. Using a charm you can scale back the number of nodes, or shutdown the service quickly. You can find out more about JuJu at:

https://juju.ubuntu.com/

Notes: Ubuntu also provides Docker, a technology that allows any application to be deployed as a container that will run on any type of server (bare-metal, virtual machine, public, and Open Stack), https://www.docker.io.