Tuesday, June 23, 2015

Teach a man to fish ...

Today somebody asked me a question which I thought warranted a blog post. For the purpose of this blog post "somebody" will be referred to as Jeff.

So, Jeff came to me with a problem. Jeff had set out to build a particular solution in AWS. During his investigations he found an off-the-shelf CloudFormation template which deployed the exact solution he wanted.

Jeff downloaded the CloudFormation template from GitHub, logged in to the the AWS management console and ran through the "Create new Stack" wizard. Jeff was on top of the world, the solution was being built in front of his very eyes and so far, all he'd had to do was a bit of googling and a few mouse clicks.

He was grinning like a cheshire cat, life was good, CloudFormation was working it's magic and he was going to be the office hero ... right up until the moment he saw the dreaded ROLLBACK_IN_PROGRESS message.

S**! Jeff thought to himself as he watched his beautiful solution torn down, volume-by-volume, instance-by-instance, ELB-by-ELB.

He opened up the CloudFormation template using his trusted copy of Sublime and this is what he saw:

"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer nec odio. Praesent libero. Sed cursus ante dapibus diam. Sed nisi. Nulla quis sem at nibh elementum imperdiet. Duis sagittis ipsum. Praesent mauris. Fusce nec tellus sed augue semper porta. Mauris massa. Vestibulum lacinia arcu eget nulla. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos."

Well, he didn't really see that, but I'm sure you can appreciate that to the untrained eye, CloudFormation templates can definitely look a little scary.

That's when Jeff decided to call me and ask for a little help. We walked through the template and tried to identify the reasons for the failure, which are beyond the scope of this post. One thing which did become clear from the silence on the other end of the line was that Jeff was struggling a little bit to keep up with my troubleshooting approach, how did we get from Error Message A to Solution B.

Jeff then reminded me of the famous quote "Give a man a fish and you feed him for a day, teach him how to fish and you teach him for life".

Now, just to put a little context around my friend Jeff, he's a very smart developer. It would not take Jeff long to "learn how to fish". But, what were the best resources to help Jeff "learn to fish".

AWS have an awesome documentation library and below I've included a few of the, in my personal opinion, best links for getting to grips with CloudFormation.

This first link is a great starting point for anyone wanting to start out with CloudFormation and understand the building blocks of a template:

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html

This next link is a the bible of AWS CloudFormation resources. It provides an invaluable breakdown of every resource type you can create through CloudFormation. Definitely my first stop when handcrafting and troubleshooting templates.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html

I'm sure I'll hear from Jeff again. But I know that armed with new arsenal of new links, he will try his absolute best to catch that fish on his own first. He may not succeed, but he will learn a lot and with each attempt, this;

"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer nec odio. Praesent libero. Sed cursus ante dapibus diam. Sed nisi. Nulla quis sem at nibh elementum imperdiet. Duis sagittis ipsum. Praesent mauris. Fusce nec tellus sed augue semper porta. Mauris massa. Vestibulum lacinia arcu eget nulla. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos."

Will start to look more like this;

  "Resources" : {
    "EC2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
        "InstanceType" : { "Ref" : "InstanceType" },
        "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
        "KeyName" : { "Ref" : "KeyName" },
        "ImageId" : "ami-2323232"
      }
    },

Wednesday, June 10, 2015

Programming monogamy

I decided it was high time I bought an end to my exclusive relationship with PHP and started playing the field a little.

I'm not suggesting that there is anything wrong with PHP. Exactly the opposite in fact, not coming from a development background, I've been able to achieve some fantastic things with PHP. My journey from "total PHP newb" to "not so much of a PHP newb" has introduced me to a new whole new world and some very interesting concepts. Starting to understand application development and some of the application programming principals has helped improve my understanding of software development within my company. It has enabled me to have more constructive conversations with developers across my organisation, this was especially relevant when it came to re-factoring some of our software solutions for AWS.

That aside, I recently met this little beauty who goes by the name of Ruby. Why Ruby and not Python or Node or some other scripting language. The main motivator was Rails. Rails is a web application framework that I've heard lots about and am keen to explore.

A lot of the PHP work I've done has been around building web applications and browser based consoles for managing environments and services within our environment. I never got really stuck into a framework for developing applications in PHP, I normally handcraft everything with a simple MVC structure, like the one below.

This practice got hammered home thanks to the cover-to-cover reading of "PHP for Absolute Beginners", which I'd highly recommend to anyone looking to get started with PHP or web application development in general.

Anyway, I digress, since Ruby and Rails "appear" to go hand-in-hand, I decided to start learning some Ruby.

I wanted to start, as I did with PHP, with something really simple. Since most of what I do these days resolves around AWS, pulling back a list of EC2 instances and dumping them to the console seemed like a perfect place to start.

In my first little script below, I've created an empty hash, built a function which uses the AWS SDK for ruby to return a list of instances and populate the hash with a subset of the information returned.

It then iterates through the hash, using the awesome .each method and spits out a "nicely" formatted report to the console.

Pretty basic, but it gave me the chance to get an grasp on some basic Ruby concepts like symbols and the awesome .each method.

These simple scripts inevitably form the building blocks for larger and more complex solutions, so sit tight and lets see where Ruby and I go from here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/ruby 

require "aws-sdk"

$instance_hash = Hash.new('Nothing New')

################################
# => Function: get_ec2_instances
# => returns all running EC2 Instances for my account.
###############################

def get_ec2_instances
 ec2 = Aws::EC2::Client.new(region: 'ap-southeast-2')

 resp = ec2.describe_instances()
 resp[:reservations].each do | reservations |
  reservations[:instances].each do | instances |
   $instance_hash[instances[:instance_id]] = 
    {
    "accountId" => reservations[:owner_id],
    "state" => instances[:state][:name], 
    "privateIp" => instances[:private_ip_address]
    }
  end
 end
end

get_ec2_instances

$instance_hash.each do |key,value|
 puts "Intstance Id: #{key}"
 value.each do |k,v|
  puts "#{k} : #{v}"
 end
 puts "-" * 25
end

Monday, June 01, 2015

Route53 + RaspberryPi + Cron + PHP =  lazy admin.

Thanks to my recently AWS-connected Pi, performing a scheduled DNS cutover from the comfort of my own bed could not have been easier.

With a little cron magic and some "Aws\Route53\Route53Client" you can easily schedule changes to Route53 records / record sets. 


<?php
error_reporting(E_ALL);
ini_set("Display Errors", 1);
require 'vendor/autoload.php';
// Create client object for Route53
$r53Client = \Aws\Route53\Route53Client::factory(array());
// Create client object for SES
$SesClient = \Aws\Ses\SesClient::factory(array(
    'region' => 'us-east-1'
));
// Function for sending notifications if the record change fails or for confirmation that the change has been made.
function StackNotification($body, $cnameDns)
{
    global $SesClient;
    $stackSubject = 'DNS Update Confirmation ' . "[$cnameDns]";
    $SesClient->sendEmail(array(
        'Source' => 'blahblah@mitchyb.com',
        'Destination' => array(
            'ToAddresses' => array(
                'blahblah@mitchyb.com'
            )
        ),
        'Message' => array(
            'Subject' => array(
                'Data' => $stackSubject
            ),
            'Body' => array(
                'Html' => array(
                    'Data' => $body
                )
            )
        )
    ));
}
function updateRecord($elbDns, $cnameDns)
{
    global $r53Client;
    global $cloudFormationStackName;
    // Update DNS Records
    try {
        $command = $r53Client->changeResourceRecordSets(array(
            'HostedZoneId' => 'Z16PRLGBWGMRUY',
            'ChangeBatch' => (object) array(
                'Changes' => (object) array(
                    array(
                        'Action' => 'UPSERT',
                        'ResourceRecordSet' => array(
                            'Name' => $cnameDns,
                            'Type' => 'CNAME',
                            'TTL' => 60 * 5,
                            'ResourceRecords' => array(
                                array(
                                    'Value' => $elbDns
                                )
                            )
                        )
                    )
                )
            )
        ));
        
        $msg = "Route53 record updated to " . $elbDns;
        StackNotification($msg, $cnameDns);
    }
    catch (Exception $e) {
        $errorMsg = "Route53 record update failed with error: $e";
        trigger_error($errorMsg);
        
        StackNotification($errorMsg);
        exit;
    }
}
;
// Call the record set update function.
updateRecord('offline.mitchyb.com', 'blog.mitchyb.com');

A little about Me

My photo
My name is Mitch Beaumont and I've been a technology professional since 1999. I began my career working as a desk-side support engineer for a medical devices company in a small town in the middle of England (Ashby De La Zouch). I then joined IBM Global Services where I began specialising in customer projects which were based on and around Citrix technologies. Following a couple of very enjoyable years with IBM I relocated to London to work as a system operations engineer for a large law firm where I responsible for the day to day operations and development of the firms global Citrix infrastructure. In 2006 I was offered a position in Sydney, Australia. Since then I've had the privilege of working for and with a number of companies in various technology roles including as a Solutions Architect and Technical team leader.