CloudBolt Blog

UPDATE: As of Verison 5.3.1, it will no  longer be necessary to check EC2 instance reachability. This functionality has been rolled into the product. This article is still a great example of what it takes to write a CloudBolt plug-in and will be useful in many other scenarios. ~Rick

A common use-case I see frequently is the need to make sure new EC2 instances are up and ready to accept SSH connections before CloudBolt marks the provisioning job as complete. In this article, we’re going to work together to write a CloudBolt plug-in that will add this functionality to our CloudBolt environments. In doing so, I hope you'll not only gain an appreciation for the power of CloudBolt as a cloud automation platform, but you'll also see how easy it is to extend our base feature set using upgrade-safe scripts.

Getting Started

Writing Python code is a relatively painless process that usually starts with a text editor. I use OSX, so I prefer TextMate. If you’re a Windows user, I suggest Sublime Text 2 (http://www.sublimetext.com/2) or Notepad++. Another great option is to use PyCharm for all your CloudBolt plug-in development projects. I plan to expand on this topic in a future article.

Planning Our Attack

Let’s talk briefly about what we want to accomplish with this plug-in: When we provision a VM to EC2 via CloudBolt, we want to wait until that server is finished initializing and ready for SSH access before marking the entire CloudBolt provisioning job as complete. By default CloudBolt marks the job complete once the VM state is set to “OK” by AWS. Unfortunately, this isn’t the full story on the VM's readiness. The “OK” state is set before the VM is initialized and before the user can login via SSH. Imagine your poor users – they just used the awesome CloudBolt platform to spin up a VM, and once their job is “complete”, they get a “Connection Refused” error when they try to connect via SSH – not cool.

To address this issue, we'll extend CloudBolt to wait until our new EC2 instance has passed all EC2 status checks before marking the job as successfully completed. To accomplish this, we’ll trigger an action at the post-provision stage of the “Provision Server” Orchestration Action that will poll EC2 every two seconds to see if our new instance is reachable according to the EC2 status checks. We‘ll implement this action as a CloudBolt plug-in script written in Python.

Starting our Plug-in

Let's start our plug-in with a file called “poll_for_init_complete.py” with the following contents:

The CloudBolt platform knows to call this function when it‘s time to execute the plug-in, therefore it's essential that it exists in your plug-in script. Note that the first and required parameter to this function is called job. This implies that we should expect the CloudBolt platform to call this function with the originating provisioning job passed as a job.models.Job object.

Returning a tuple of ("", "", "") is the default way of communicating to the CloudBolt platform that the script was a success.

Let's Get Busy

Let's add a few more lines to our plug-in script to get the server (our new EC2 instance) from the Job object and wait until it's reachable:

Let's walk through what what we have so far:

server = job.server_set.first() sets the variable called server to the Server object associated with this job. Since we're working with a server provisioning job, it's safe to assume we're only going to have one Server associated with this job, therefore we call first() on our job's server_set property.

We defined a constant called TIMEOUT in our plug-in module and set it to 600. We then use this TIMEOUT at timeout = time.time() + TIMEOUT to set the time at which we should no longer wait for our EC2 instance to initialize. This prevents CloudBolt from waiting indefinitely if for some reason EC2 cannot determine the reachability of our server. Since this is in seconds, we'll stop waiting after a maximum of 10 minutes has passed before marking the job as complete. This should be the exception – not the norm.

We then start an infinite loop that will only stop when either our timeout elapses or we determine that our EC2 instance is reachable with the function is_reachable(server) which we haven't yet defined.

Is it Reachable or Not?

The script above is still missing the implementation of our is_reachable function. Given the server object associated with this job, this function will use the AWS Boto API to determine the reachability status for our new EC2 instance.  Note: Boto is the name of the Python API used to access the AWS API.

Let's add our is_reachable function to our script above our run function:

Let's step through this function step-by-step:

1. instance_id = server.ec2serverinfo.instance_id
   Get the EC2 instance ID associated with our new server being provisioned through CloudBolt. This is a string that looks like i-2423c494 in the EC2 console.

2. ec2_region = server.ec2serverinfo.ec2_region
   Get the AWS region into which our new EC2 instance is being deployed.

3. A few CloudBolt platform API gymnastics to get the backing Boto API objects without specifying any credentials. Always keep credentials out of your scripts!
   rh = server.resource_handler.cast()
rh.connect_ec2(ec2_region)
wc = rh.resource_technology.work_class

4. instance = wc.get_instance(instance_id)
   Get the Boto Instance object associated with our new server's instance ID.

5. status = instance.connection.get_all_instance_status(instance_id)
   Using the connection associated with our Boto Instance object, return the instance status for our server.

6. return True if status[0].instance_status.details[u'reachability'] == u'passed' else False
   If the reachability status for our server is “passed”, return True because our new server is now reachable. If not, return False. We use status[0] because our get_all_instance_status function above returns an array. In this case we're only asking for the status of one instance, so we know the array only has one Status object and thus we use status[0].

Going back to our loop you can now see how the is_reachable function is used to keep the loop going if the answer is false:

If our server is NOT reachable, and our timeout hasn't expired, we wait two seconds and try again.

Putting it All Together

The complete script can be downloaded from cloudbolt-forge. 

Now that it's ready, let's add it to the appropriate trigger point in CloudBolt.

In your CloudBolt instance, navigate to Admin > Actions > Orchestration Actions and click “Provision Server” on the left tab bar. Find the “Post-Provision” trigger point at the bottom of the page and click the “Add an Action” button.

Select “CloudBolt Plug-in” and in the next dialog, click "Add new cloudbolt plug-in".

Specify a name for our new plug-in (Poll for EC2 Init Complete), select the "Amazon Web Services" resource technology, browse to your script, and click "Create".  Selecting the "Amazon Web Services" resource technology ensures this plug-in only runs against AWS resource handlers that you've defined and not others to which this plug-in is not applicable.

Give it a try

Provision a server to one of your AWS-backed CloudBolt environments. Watching the job progress, you'll see that the job is not marked as complete until the server is fully reachable and SSH access is available.

Questions? Comments? Concerns?

Don't hesitate to reach out to me (rkilcoyne@cloudbolt.io) or any of the CloudBolt Solutions team for help!

In our latest release, we've continued enabling IT organizations that want to provision and manage their applications more effectively. CloudBolt v4.6 makes providing self-service IT access to applications easier than ever, regardless of whether that application resides on a single server, or is a complete end-to-end stack of servers deployed across several environments. Like never before, users can interactively request entire stacks with just a few clicks, and deploy those stacks into any one of the dozen or so supported cloud and virtualization platforms.

We haven’t stopped there, though. In addition to streamlining the application provisioning process, we’ve put a significant amount of effort into other areas of CloudBolt as well. 

Nebula

We're also proud to announce an all-new connector for Nebula Private Cloud environments.

With this new connector, CloudBolt customers gain the ability to deploy into and manage servers, applications, and even entire services in Nebula-backed environments. Nebula private cloud customers using CloudBolt gain immediate access to all of CloudBolt's features in both new and existing environments:

• Chargeback and Showback
• Reporting
• Governance
• Automated provisioning and management
• Lifecycle management
• Software license management
• And more

Service Catalog

Customers are using the CloudBolt Service Catalog to provide end users self-service access to entire application stacks for some time now. In v4.6, we've updated the service creation process to make it even more straightforward. Just as they can do for the single server ordering process, admins can alter how the service ordering process looks for different end users and deployment environments. End users can be prompted to enter specific information as necessary based on their desired target deployment environment. 

Once ordered by a user, CloudBolt’s built-in approval mechanism can be leveraged for additional validation before CloudBolt steps through any number of automated processes required for delivery of a fully functional application stack.

The end result is clear: CloudBolt administrators can quickly create new service offerings that are able to span any supported target environment. Regardless of your platform of choice, CloudBolt can deliver a complete stack to your end users, and in less time than you think. 

Active Directory Group Mapping

Are you using one or more AD environments to authenticate CloudBolt users? In v4.6, admins gain the ability to map AD groups to CloudBolt groups. This AD group mapping also works with multiple AD environments, so if you're using CloudBolt in a multi-tenant capacity, you can still pick-and-choose how auth is handled for each tenant. 

Orchestration Hooks

Orchestration Hooks enable IT administrators to automate every step needed to deliver IT resources and applications to end-users. Extending on this capability, we've added a new Orchestration Hook type that enables the execution of an arbitrary remote script.

This further extends CloudBolt’s lead as the most powerful cross-platform application deployment and management platform, as it can now be seamlessly integrated into nearly any manually-scripted provisioning and management process. Reusing your existing IP has never been easier or faster. 

Connector Improvements

Discovering and importing current state from existing environments is a CloudBolt Cloud Management key strength. In v4.6, this is even more thorough, as we now also detect all disk information from VMware vCenter virtual machines as well as AWS AMI, and Microsoft Azure public cloud instaces. 

Have a lot of VMs? Users in environments with tens of thousands of VMs will be happy to learn that VM discovery and sync is more efficient and faster. 

Puppet Enterprise users can now also leverage multiple Puppet environments rather than the default "Production". This can further help customers simplify their IT environments. 

Get It Now

The CloudBolt Cloud Manager v4.6 is available today via the CloudBolt support portal. Our updates are just another feature, and take mere minutes to complete. 

Don't have CloudBolt yet?

Introducing the latest release of CloudBolt C2: v4.5

Connector Updates

With C2 v4.5, we’ve added two new connectors that further expand the breadth of technologies IT organizations can manage from a single-pane-of glass.

Google Compute Engine support gives administrators the ability to seamlessly offer end users controlled access to yet another public cloud provider. This includes the ability to install and manage applications from a supported configuration manager, as well as the ability to include GCE instances in C2 Service Catalog service templates.

C2 v4.5 includes support for Google Compute Engine in the Google Cloud Platform.

We’ve also totally re-written and re-based our OpenStack connector. In this update, we’ve focused on compatibility, and we’re now able to support Icehouse, Havana, and Grizzly from the major OpenStack providers such as Mirantis. Of course, C2 can include OpenStack-backed resources when provisioning applications, running external flows, and accounting for licenses, just to name a few. C2 is already the best dashboard for OpenStack, and it’s getting even better with each release. No Horizon development needed!

We’ve also made some additional updates to our vCenter connector, including improved error handling when your VMware tools are out of date, and allowing for longer Windows hostnames. We’ve also made the Windows disk extending messages more clear and straightforward.

Amazon Web Services has also received some developer love. C2 now synchronizes both the public and private IP addresses for each AWS EC2 instance.

Configuration Management

We worked closely with the engineering team at Puppet, and now have a unique capability: C2 can now discover and import classes from a Puppet server.

Chef integration is even better: C2 now enables Chef bootstrapping on Windows and Ubuntu Linux systems.

User Interface Updates

Updates to the C2 UI are perhaps more subtle, but focused on helping users and administrators more effectively manage large numbers of applications and servers. We’ve integrated simple indicators describing the total number of selected items in each data table, making it much easier to manage large environments.

Did you know that you can use C2 to open network-less console connections on C2-managed servers? We’ve made this feature faster and more reliable in C2 v4.5.

Upgrading

Upgrading C2 is just like any other feature in C2: fast, easy, and predictable. Upgrading to C2 v4.5 is now even faster and easier than before.

Sounds Great, I Want It!

CloudBolt C2 has been recognized by Gartner for our industry-leading time to value. We effectively eliminate the barrier to entry for enterprise Cloud Management. C2 v4.5 is available today. Request a download, and you'll be up and running in your own environment in no time.