This document discusses monitoring tools for Amazon AWS cloud computing environments. It compares the open source Hyperic HQ tool, which requires installing agents on EC2 instances, to Amazon CloudWatch, which has no installation requirements. While Hyperic HQ enables more comprehensive and automated monitoring, CloudWatch integrates better with AWS and has lower costs for usage-based pricing. The author concludes that using both tools provides the best monitoring solution currently for complex AWS production systems.

1. Monitoring on Amazon AWS Cloud
Cloud computing concepts like on-demand scalability, high-availability and usage based pricing
models have pushed real-time monitoring from the side alleys onto center stage. Monitoring
tools have become an absolute necessity and constitutes an important part of any
distributed/cloud application. It is a challenge for an AWS Cloud administrator to manage and
efficiently monitor these new breed of elasticity in their production environment.
An ideal monitoring tool on AWS cloud must be
 Highly powerful, with minimum (or no) configuration to get started.
 Have minimum (or no) impact on the monitored machines' device utilization.
 Consume minimal network bandwidth during data collation (as network bandwidth
in the cloud comes with a cost overhead).
 Support monitoring performance parameters across different types of resources
including the app server, database, network, etc and AWS cloud resources like
Elastic cloud compute EC2 ,Simple storage Service S3 , Simple Queue Service SQS ,
Amazon RDS , Elastic Block storage EBS etc
 Provide access to all the metrics with user based dashboard views, reports, APIs
As part of our work on Cloud Computing Consulting on 8KMiles, we constantly research and try
out various monitoring tools continuously. The top 2 monitoring tools that we use for our
internal systems are Hyperic HQ and Amazon CloudWatch. Let us see what they are and how
they compare in broad level:
Cloud Consulting Services
Copyright 8KMiles 2009 All rights reserved www.8kmiles.com/cloud

2. Hyperic HQ
Hyperic HQ is an open source systems monitoring, server monitoring, and IT management
software. Hyperic is easy to deploy and configure. It automatically discovers, monitors, and
manages software and network resources. It provides a single pane view of performance and
availability of apps running on any OS. It also ensures apps are meeting their SLAs through
alerts and automated corrective actions.
Hyperic is built for web operations making web-based applications more manageable and to
ease the burden on the web operations team. Hyperic HQ is a Java-based platform for
monitoring and managing software resources. The key elements of the architecture are the HQ
Server, which provides centralized management and persistence, and the HQ Agent, which
provides monitoring and control on a per-monitored-platform basis.
You run an HQ Agent on each machine you want to manage with Hyperic HQ. Upon first
startup, the agent auto-discovers the software running on the EC2 instance, and periodically re-
scans for configuration changes. HQ Agents gather availability, utilization, performance, and
throughput metrics; perform log and event tracking; and allow you to perform software control
actions, for instance, starting and stopping web and application servers. Agents send the
inventory data and metrics they collect to a central HQ Server.
The HQ Server receives inventory and metric data from HQ Agents and stores it in the HQ
database. The server provides facilities for managing your software inventory—it implements
the HQ inventory and access model, grouping your software assets in useful ways to
ease monitoring and management.
Cloud Consulting Services
Copyright 8KMiles 2009 All rights reserved www.8kmiles.com/cloud

3. Amazon CloudWatch
Amazon CloudWatch is an easy-to-use web service that provides comprehensive monitoring for
Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing (ELB), and Amazon
Relational Database Service (Amazon RDS) and Amazon Elastic Block Storage (EBS). It’s very
easy to get started with Amazon CloudWatch, there is no upfront setup in order to start getting
the metrics.
Amazon CloudWatch provides the data you need to make informed decisions. You specify the
metric you want and you receive statistics for that metric. This allows you to build complex
queries in order to access different views of the data, for periods of one-minute up to two
weeks in length.
Amazon EC2 Instance Amazon Elastic Load Amazon RDS metrics Amazon EBS Volume
Metrics Balancing Metrics metrics
CPUUtilization Latency CPUUtilization VolumeReadBytes
NetworkIn RequestCount FreeStorageSpace VolumeWriteBytes
NetworkOut HealthyHostCount DatabaseConnections VolumeReadOps
DiskWriteOps UnHealthyHostCount ReadIOPS VolumeWriteOps
DiskReadBytes WriteIOPS VolumeTotalReadTime
DiskReadOps ReadLatency VolumeTotalWriteTime
DiskWriteBytes WriteLatency VolumeIdleTime
ReadThroughput VolumeQueueLength
WriteThroughput
Information about the metrics is available in Amazon CloudWatch developer guide
With Amazon CloudWatch, you only pay for what you use. Amazon CloudWatch is charged on
hourly basis by the number of Amazon EC2 instances we monitor. There is no charge for
monitoring EBS Volumes, Elastic Load Balancers or RDS Database Instances. Example, a Cloud
administrator may want to monitor 10 Amazon EC2 instances 24×7 for a 30-day period. The
Amazon CloudWatch cost would be $108 (or $0.015 per Amazon EC2 instance hour x 10
Amazon EC2 instances x 24 hours per day x 30 days). Partial hours will be billed as full hours.
(Refer Amazon CloudWatch page for latest rates and updates)
Cloud Consulting Services
Copyright 8KMiles 2009 All rights reserved www.8kmiles.com/cloud

4. Hyperic HQ Vs Amazon CloudWatch
 Hyperic needs installation of HQ Agent on all the EC2 instances to be monitored and HQ
server on an EC2 instance to monitor and manage all the other machines. Amazon EC2
needs no software to be installed. Monitoring needs to be enabled to collect all the
metrics of the instances, Elastic load balancer, EBS and Amazon RDS.
 With Hyperic it’s easy to setup alerts and take automated corrective action. Amazon
CloudWatch does not support alerts and automated corrective action feature as of now.
We have to develop ourselves the alert and notifications using the AWS CloudWatch
API.
 Hyperic monitors can monitor databases, web and application servers, middleware,
network devices, etc. Amazon EC2 monitors only the specific set of metrics mentioned
in the above table. Hyperic is not very efficient in monitoring Amazon RDS and Amazon
ELB.
 Hyperic HQ open source edition is free and Amazon CloudWatch is charged on usage
basis. But we have to pay for the EC2 Instance on which Hyperic HQ server is running
even though it is open source.
 Configuring Hyperic HQ on AWS cloud is complex compared configuring Amazon
CloudWatch on AWS.
 Introducing Hyperic HQ in the middle of our production environment life cycle will need
re bundling of your new images in Amazon for reusability purpose; Whereas AWS
CloudWatch can be introduced and removed at any time with ease.
 Amazon CloudWatch must be enabled if we need Auto Scaling features of AWS.
 Monitored Data is persisted for 2 weeks in CloudWatch even if the monitored instances
are terminated. In Hyperic HQ it can be persisted as much time as we want.
At 8KMiles, we use both Hyperic HQ Open Source and Amazon CloudWatch to monitor our
production setup. We have provided assistance and consultation service to many of our
customers to configure, manage and monitor their production systems using Hyperic HQ and
Amazon CloudWatch on AWS. We feel that the blend of both these tools is required to monitor
the production systems at this stage because Amazon CloudWatch is in its evolution stage and
it is not a comprehensive monitoring Service which will help the Cloud System administrator to
monitor a complex production system involving Queues, Amazon MapReduce , Notifications ,
Simple DB and S3. But we expect to see more features added into Amazon CloudWatch in
coming months and we are sharing our inputs and experiences with Amazon AWS team
constantly on this front.
Cloud Consulting Services
Copyright 8KMiles 2009 All rights reserved www.8kmiles.com/cloud

5. Author
Santhosh D is a Cloud Engineer at 8KMiles specialized in Cloud Infrastructure and Developing
Large Scale Web Apps using Cloud Computing. He is a vital player of the Virtual Computing
Environment team where he integrates the AWS Cloud services into 8KMiles platform.
Santhosh helps companies to get started on Amazon AWS and then fall in love with it.
About 8KMiles
8KMiles is an internet company that is focused on building solutions around cloud computing.
8KMiles Cloud Solutions group offers cloud consulting, engineering and Cloud migration
services to help companies leverage the power of cloud computing. Our team of AWS Cloud
experts – located in North America and India provides Cloud Consulting, Migration, Remote
infrastructure management, Backup, Auto Scaling, GEO Load balancing and Disaster Recovery
services over AWS.
For more information, visit www.8kmiles.com/cloud
Contact
India
Harish Ganesan
Co Founder and CTO
Harish@8KMiles.com
USA
Paddy
Co Founder and Chief Evangelist
paddy@8KMiles.com
1-408-647-1217
Cloud Consulting Services
Copyright 8KMiles 2009 All rights reserved www.8kmiles.com/cloud