VMware : Building an AMI – best practices

This blog post is co-authored by Soumay Das, Anish Swaminathan, Georgi Alexandrov, Dean Efrati, and Thiru Bhat

AMI is Amazon Machine Image which provides the information needed to launch an instance in AWS. In this article, we look at building a custom AMI for specific service needs and the best practices when building a custom AMI. It is possible to take an existing AMI, make changes to it and save it for the future. It is also possible to bring ones own OS and save the AMI. We will cover both the scenarios in this document.

As noted in the AWS article, the AWS AMI lifecycle is illustrated below:

1.1. Taking an existing image and customizing it for later usage:

AWS allows an EC2 instance to be launched with a custom machine image also known as Amazon Machine Image. (AMI)

In the quick start section, there are several AMIs available for different needs, or one can create ones own image in a region and save it for future use, pick one from AWS Marketplace or choose a Community AMI. We only detail taking an existing image and customizing it for later usage.

Building a custom AWS AMI:

An AWS AMI can be built using packer, a tool provided by HashiCorp. For example, it is possible to take a market place AMI with the latest patches such as CentOS 7 HVM AMI. Then one can install custom packages on it such as the Nessus Agent, specific packages such as terraform, curl, telegraf, wavefront, python, wget, jq, specific version of kubernetes binaries based on need. The AMI is region specific and the steps to copy the image is noted in this page.

Detailed instructions to build an AWS AMI for EKS are also available through AWS as noted here.

Custom AWS AMIs are desirable in these situations:

1. There is a security, compliance need. This needs specific tools or packages to be pre-installed on the image. An example is the installation of wazuh agent on the machine image. In this case, it is desirable to customize a machine image and save it for all the micro services to re-use.
2. A well tested, updated image with the right patches is available. In this case, it is logical to save this image as an AMI and re-use it for various services with additional packages. The packages can be installed after the instance is running in EC2 as noted here. In case there is a need for faster start-up, a separate AMI can be created with the packages pre-installed on top of the base image. There is a trade off between having too many AMIs vs having a base image that is customized at run time.

It is a best practice to keep the number of additional packages to a minimum. This helps reduce the security attack surface and also minimizes the need to install patches for security updates. Once the AMI is part of a running EC2 instance, there is a need to have a policy to periodically patch the OS with security updates. The updates can be a blue-green update or a rolling update.

1.2. Creating a brand new image using Photon OS:

Photon OS is a light weight linux operating system optimized for Cloud Native workloads. More details on Photon OS noted in this link and the AMI images can be downloaded from this location. Photon OS is multi cloud ready with images running in private cloud (vsphere), AWS, GCP and Azure. This blog post focuses on AWS.

An AMI image can be created, used with a snapshot as noted in the AWS page. One can create a snapshot of the Photon OS image using Amazon Linux. Amazon Linux has all the AMI tools installed in it.

Building an Amazon Machine Image (AMI)

Overview

An Amazon AMI (Amazon Machine Image) is an image with the information needed to launch an instance on AWS (Amazon Web Services). An AMI contains the software configuration needed for the instance, for example, an application server, an operating system, and applications.

This article addresses the use case of building a custom AMI for specific service needs, and the best practices for building such AMIs. Note that you can make changes to an existing AMI and save the changes for future use.

Photon OS and Lightwave AMI

Photon OS platform comes with several pre-packaged applications. The scripts currently build OVAs that you can deploy on VMware environments. However, for non-VMware environments, you need to build other image formats. The following section walks you through the steps for building an AMI for VMware Lightwave with Photon OS as the operating system.

Building the Base Photon OS AMI

Presently, there is no suitable AMI published for Photon OS. Therefore, you must build a Photon OS AMI. This section demonstrates the steps to build an EBS-backed AMI with Photon OS as the base. An EBS-backed AMI ensures the availability of a wide range of instance types for use.

1. Deploy an Amazon Linux instance to the region of your choice. For example, us-west-2.

Note: You can use any Linux-based instance. However, we recommend an Amazon Linux instance for the least resistance.

2. SSH into the deployed instance.

Note: The default user on Amazon Linux is "ec2-user".

3. If you are not using an Amazon Linux instance, follow the steps to install the required AMI tools from the AMI tools

Note: Amazon Linux instance has all the required AMI tools pre-installed.

4. Download the AMI binary from the Downloading Photon OS

Note: In this example, we have used Photon AMI 1.0.

[ec2-user@ip-xxx ~]$ wget https://bintray.com/artifact/download/vmware/photon/photon-ami-1.0-13c08b6.tar.gz [ec2-user@ip-xxx ~]$ tar -xzf photon-ami-1.0-13c08b6.tar.gz

5. Create a private and public key pair.

$ openssl genrsa 2048 > private-key.pem $ openssl req -new -x509 -nodes -sha256 -days 365 -key private-key.pem -outform PEM -out certificate.pem

6. Set your AWS credentials.

[ec2-user@ip-xxx ~]$ aws configure AWS Access Key ID [None]: AWS Secret Access Key [None]: Default region name [None]: us-west-2 Default output format [None]:

7. Create an EBS volume, and then attach the EBS volume to your instance.

[ec2-user@ip-xxx ~]$ ec2-metadata | grep instance-id instance-id: i-05b0bf990a6f329df [ec2-user@ip-xxx ~]$ aws ec2 create-volume -size 8 -region us-west-2 -availability-zone us-west-2b { "AvailabilityZone": "us-west-2b", "Encrypted": false, "VolumeType": "standard", "VolumeId": "vol-00334b7296d76fb5c", "State": "creating", "SnapshotId": "", "CreateTime": "2017-03-13T23:14:51.483Z", "Size": 8 } [ec2-user@ip-xxx ~]$ aws ec2 attach-volume -volume-id vol-00334b7296d76fb5c -instance-id i-05b0bf990a6f329df -device /dev/xvdb -region us-west-2 { "AttachTime": "2017-03-13T23:16:30.580Z", "InstanceId": "i-05b0bf990a6f329df", "VolumeId": "vol-00334b7296d76fb5c", "State": "attaching", "Device": "/dev/xvdb" }

8. Copy the RAW image file of the Photon AMI to the attached disk.

[ec2-user@ip-xxx ~]$ dd if=photon-ami-1.0-13c08b6.raw of=/dev/xvdb bs=1M 8192+0 records in 8192+0 records out 8589934592 bytes (8.6 GB) copied, 406.188 s, 21.1 MB/s

9. Mount the disk.

[ec2-user@ip-xxx ~]$ sudo partprobe /dev/xvdb [ec2-user@ip-xxx ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdb 202:32 0 8G 0 disk ├─xvdb1 202:33 0 2M 0 part └─xvdb2 202:34 0 8G 0 part [ec2-user@ip-xxx ~]$ sudo mkdir /mnt/ebs [ec2-user@ip-xxx ~]$ sudo mount /dev/xvdb2 /mnt/ebs

10. Remove the resolv.conf file, if it exists.

[ec2-user@ip-xxx ~]$ sudo rm /mnt/ebs/etc/resolv.conf

11. Unmount the disk and then create a snapshot.

[ec2-user@ip-xxx ~]$ sudo umount /mnt/ebs [ec2-user@ip-xxx ~]$ aws ec2 detach-volume -volume-id vol-00334b7296d76fb5c -region us-west-2 { "AttachTime": "2017-03-13T23:16:30.000Z", "InstanceId": "i-05b0bf990a6f329df", "VolumeId": "vol-00334b7296d76fb5c", "State": "detaching", "Device": "/dev/xvdb" } [ec2-user@ip-xxx ~]$ aws ec2 create-snapshot -region us-west-2 -description "photonOS ebs ami" -volume-id vol-00334b7296d76fb5c { "Description": "photonOS ebs ami", "Encrypted": false, "VolumeId": "vol-00334b7296d76fb5c", "State": "pending", "VolumeSize": 8, "Progress": "", "StartTime": "2017-03-13T23:27:29.000Z", "SnapshotId": "snap-0e3a12eea11ccca46", "OwnerId": "367199020685" } [ec2-user@ip-xxx ~]$ aws ec2 describe-snapshots -region us-west-2 -snapshot-id snap-0e3a12eea11ccca46 { "Snapshots": [ { "Description": "photonOS ebs ami", "Encrypted": false, "VolumeId": "vol-00334b7296d76fb5c", "State": "pending", "VolumeSize": 8, "Progress": "0%", "StartTime": "2017-03-13T23:27:29.000Z", "SnapshotId": "snap-0e3a12eea11ccca46", "OwnerId": "367199020685" } ] } … wait a few minutes [ec2-user@ip-xxx ~]$ aws ec2 describe-snapshots -region us-west-2 -snapshot-id snap-0e3a12eea11ccca46 { "Snapshots": [ { "Description": "photonOS ebs ami", "Encrypted": false, "VolumeId": "vol-00334b7296d76fb5c", "State": "completed", "VolumeSize": 8, "Progress": "100%", "StartTime": "2017-03-13T23:27:29.000Z", "SnapshotId": "snap-0e3a12eea11ccca46", "OwnerId": "367199020685" } ] }

12. Register the AMI.

[ec2-user@ip-xxx ~]$ aws ec2 register-image -region us-west-2 -name photon-ebs -root-device-name /dev/xvda -block-device-mappings DeviceName=/dev/xvda,Ebs={SnapshotId=snap-0e3a12eea11ccca46} -virtualization-type hvm -architecture x86_64 { "ImageId": "ami-d20d83b2" }

13. Deploy and test the AMI.

Note: Default user on Photon OS is "root".

Building the Lightwave Application AMI

Pre-requisite: Ensure that you have already checked out the photon-controller code, and you are in the root directory.

Execute the following command:

$ cd appliances/lightwave $ packer build -force -only=amazon-ebs -var 'aws_access_key= -var 'aws_secret_key= -var 'aws_subnet_id=' -var 'aws_vpc_id=' lightwave-packer.json

To summarize, AWS provides multiple ways to build and use a custom machine image based on user needs. We have illustrated a couple of methods, one using VMware's Photon OS.