CloudBolt Software Logo

CloudBolt Blog

On Cloud Management Platforms and Interchangeable Parts

Posted by Bernard Sanders

3/11/16 7:37 AM

"A little piece of advice. You see an Agent, you do what we do. Run. You run your a** off."
Cypher, The Matrix

If you are evaluating Cloud Management Platforms (CMPs), make sure to consider whether the solution requires an agent on the VMs it manages.  This is important because your CMP should provide visibility and manageability across all servers, not just the ones it built itself.  CMPs that require an agent for full management greatly inhibit your ability to on-board previously-existing (aka “brownfield”) servers.  Consider the challenge of getting root/Administrator access to all VMs across all environments and installing software on them.  Also, test what happens when people provision VMs outside of the CMP by going straight to vCenter, AWS, etc.  Will the tool automatically discover and manage them, or will an administrator need to manually install agents first?  Also consider which OS versions the agent is supported on, how quickly new agent versions are released to support new OS versions, and what the upgrade process is for existing agents.
Agent.png

Inclusion of an agent makes a CMP more like a monitoring or configuration management system.  It blurs an important line between monitoring and operations.  Agent installation and inter-process communications also raise legitimate security concerns.  By contrast, a product designed as a manager-of-managers gets all the information it needs through the APIs of each virtualization system, public cloud, and configuration manager.   The CMP automatically discovers and manages VMs built with other tools in exactly the same way as servers it builds.  All systems are equal.

This was our design philosophy for CloudBolt.  We recognized that enterprise datacenters have a plethora of pre-existing interfaces and tools.  Rather than supplanting or requiring changes to existing operational standards, we complement and integrate with them.  Our approach enables IT architects to choose the best set of tools for each specific function.

I would go so far as to say that if a product uses agents, it is not a CMP.  A management ecosystem should be comprised of interchangeable parts and components that respect their boundaries, and integrate with each other over published, documented APIs.  Administrators should be able to swap out any particular monitoring (or backup, virtualization, or even CMP) solution for another as requirements change or better versions come along, all without affecting other functional areas.  This is central to CloudBolt's approach, and it’s one of the reasons it is consistently ranked highest amongst CMPs.

Download CloudBoltand see for yourself.

Read More

Topics: CMP, Cloud Management, Implementation, CloudBolt

CloudBolt Now Available in Azure Marketplace

Posted by Ephraim Baron

1/13/16 8:00 AM

CloudBolt’s simply powerful cloud management platform has always been available as a virtual appliance.  We enable you to manage your virtualization, private cloud, and public cloud environments all in one place.  Because our customers work in multi-cloud environments, though, we’ve received multiple requests to run CloudBolt on-demand in the public cloud.  We listened, and we’re pleased to announce CloudBolt availability in the Microsoft Azure Marketplace as a pay-as-you-go application.

Microsoft Azure Cloud

To get started, go to https://azure.microsoft.com/marketplace and search for CloudBolt.  Click on the CloudBolt logo and you’ll be presented with two options:

CloudBolt Free 25 VM Pack is a Bring Your Own License version that’s free for non-production use for up to 25 virtual machines.  You pay only for your Azure instance time.  Otherwise, it’s free to use, forever.  Whether you’re just beginning with multi-cloud management or you’re testing a variety of CMP products, this is a great way to get started with CloudBolt.  All you have to do is pick your Azure instance, request a license by return email, and follow the quick installation guide.  You’ll be up and running in minutes.

CloudBolt 125 VM Pack is an on-demand version for managing up to 125 virtual machines.  You pay only while the instance is running, and usage is billed through your Azure account.  The license is built in.  You pay a low hourly rate along with your Azure usage.  Simply pick your Azure instance and follow the quick installation guide.  Before you know it, you’ll have powerful cloud management, IT automation, user self-service, and usage/chargeback reporting at your fingertips.

CloudBolt listing in Azure Marketplace

If you have more than 125 VMs – as most of our customers do – contact us at info@cloudbolt.io.  We can easily manage thousands of VMs across a wide range of virtualization and cloud environments from a single CloudBolt instance.  And we work with legacy, brownfield environments as well as new, greenfield deployments.  All of which makes CloudBolt the central console for management, security, reporting, and control of your entire IT infrastructure.

Cloud computing is all about ease of consumption.  By offering CloudBolt in Azure, we’re making it that much simpler to manage multiple clouds from the cloud.

CloudBolt = Flexibility + Control

Read More

Topics: Partner, CMP, azure, CloudBolt

Create a CloudBolt Plug-in: Check EC2 Instance Reachability

Posted by Rick Kilcoyne

11/2/15 11:30 AM

UPDATE: As of Verison 5.3.1, it will no  longer be necessary to check EC2 instance reachability. This functionality has been rolled into the product. This article is still a great example of what it takes to write a CloudBolt plug-in and will be useful in many other scenarios. ~Rick

 

A common use-case I see frequently is the need to make sure new EC2 instances are up and ready to accept SSH connections before CloudBolt marks the provisioning job as complete. In this article, we’re going to work together to write a CloudBolt plug-in that will add this functionality to our CloudBolt environments. In doing so, I hope you'll not only gain an appreciation for the power of CloudBolt as a cloud automation platform, but you'll also see how easy it is to extend our base feature set using upgrade-safe scripts.

Getting Started

Writing Python code is a relatively painless process that usually starts with a text editor. I use OSX, so I prefer TextMate. If you’re a Windows user, I suggest Sublime Text 2 (http://www.sublimetext.com/2) or Notepad++. Another great option is to use PyCharm for all your CloudBolt plug-in development projects. I plan to expand on this topic in a future article.

Planning Our Attack

Let’s talk briefly about what we want to accomplish with this plug-in: When we provision a VM to EC2 via CloudBolt, we want to wait until that server is finished initializing and ready for SSH access before marking the entire CloudBolt provisioning job as complete. By default CloudBolt marks the job complete once the VM state is set to “OK” by AWS. Unfortunately, this isn’t the full story on the VM's readiness. The “OK” state is set before the VM is initialized and before the user can login via SSH. Imagine your poor users – they just used the awesome CloudBolt platform to spin up a VM, and once their job is “complete”, they get a “Connection Refused” error when they try to connect via SSH – not cool.

To address this issue, we'll extend CloudBolt to wait until our new EC2 instance has passed all EC2 status checks before marking the job as successfully completed. To accomplish this, we’ll trigger an action at the post-provision stage of the “Provision Server” Orchestration Action that will poll EC2 every two seconds to see if our new instance is reachable according to the EC2 status checks. We‘ll implement this action as a CloudBolt plug-in script written in Python.

Starting our Plug-in

Let's start our plug-in with a file called “poll_for_init_complete.py” with the following contents:


def run(job, logger=None, **kwargs):
    return """"""

The CloudBolt platform knows to call this function when it‘s time to execute the plug-in, therefore it's essential that it exists in your plug-in script. Note that the first and required parameter to this function is called job. This implies that we should expect the CloudBolt platform to call this function with the originating provisioning job passed as a job.models.Job object.

Returning a tuple of ("", "", "") is the default way of communicating to the CloudBolt platform that the script was a success.

Let's Get Busy

Let's add a few more lines to our plug-in script to get the server (our new EC2 instance) from the Job object and wait until it's reachable:


import time
from jobs.models import Job
 
TIMEOUT = 600
 
def run(job, logger=None, **kwargs):
    server = job.server_set.first()
    timeout = time.time() + TIMEOUT
 
    while True:
        if is_reachable(server):
            job.set_progress("EC2 instance is reachable.")
            break
        elif time.time() > timeout:
            job.set_progress("Waited {} seconds. Continuing...".format(TIMEOUT))
            break
        else:
            time.sleep(2)
 
    return """"""

Let's walk through what what we have so far:

server = job.server_set.first() sets the variable called server to the Server object associated with this job. Since we're working with a server provisioning job, it's safe to assume we're only going to have one Server associated with this job, therefore we call first() on our job's server_set property.

We defined a constant called TIMEOUT in our plug-in module and set it to 600. We then use this TIMEOUT at timeout = time.time() + TIMEOUT to set the time at which we should no longer wait for our EC2 instance to initialize. This prevents CloudBolt from waiting indefinitely if for some reason EC2 cannot determine the reachability of our server. Since this is in seconds, we'll stop waiting after a maximum of 10 minutes has passed before marking the job as complete. This should be the exception – not the norm.

We then start an infinite loop that will only stop when either our timeout elapses or we determine that our EC2 instance is reachable with the function is_reachable(server) which we haven't yet defined.

Is it Reachable or Not?

The script above is still missing the implementation of our is_reachable function. Given the server object associated with this job, this function will use the AWS Boto API to determine the reachability status for our new EC2 instance.  Note: Boto is the name of the Python API used to access the AWS API.

Let's add our is_reachable function to our script above our run function:


import time
 
TIMEOUT = 600
 
def is_reachable(server):
    instance_id = server.ec2serverinfo.instance_id
    ec2_region = server.ec2serverinfo.ec2_region
 
    rh = server.resource_handler.cast()
    rh.connect_ec2(ec2_region)
    wc = rh.resource_technology.work_class
 
    instance = wc.get_instance(instance_id)
    conn = instance.connection
    status = conn.get_all_instance_status(instance_id)
    return True if status[0].instance_status.details[u'reachability'] == u'passed' else False
 
 
def run(job, logger=None, **kwargs):
    # SNIP... 

Let's step through this function step-by-step:

  1. instance_id = server.ec2serverinfo.instance_id
    Get the EC2 instance ID associated with our new server being provisioned through CloudBolt. This is a string that looks like i-2423c494 in the EC2 console.

  2. ec2_region = server.ec2serverinfo.ec2_region
    Get the AWS region into which our new EC2 instance is being deployed.

  3. A few CloudBolt platform API gymnastics to get the backing Boto API objects without specifying any credentials. Always keep credentials out of your scripts!
    rh = server.resource_handler.cast()
    rh.connect_ec2(ec2_region)
    wc = rh.resource_technology.work_class

  4. instance = wc.get_instance(instance_id)
    Get the Boto Instance object associated with our new server's instance ID.

  5. status = instance.connection.get_all_instance_status(instance_id)
    Using the connection associated with our Boto Instance object, return the instance status for our server.

  6. return True if status[0].instance_status.details[u'reachability'] == u'passed' else False
    If the reachability status for our server is “passed”, return True because our new server is now reachable. If not, return False. We use status[0] because our get_all_instance_status function above returns an array. In this case we're only asking for the status of one instance, so we know the array only has one Status object and thus we use status[0].

Going back to our loop you can now see how the is_reachable function is used to keep the loop going if the answer is false:


while True:
    if is_reachable(server):
        job.set_progress("EC2 instance is reachable.")
        break
    elif time.time() > timeout:
        job.set_progress("Waited {} seconds. Continuing...".format(TIMEOUT))
        break
    else:
        time.sleep(2)

If our server is NOT reachable, and our timeout hasn't expired, we wait two seconds and try again.

Putting it All Together

The complete script can be downloaded from cloudbolt-forgeCloudBolt Forge is a source of user-contributed actions and plug-ins

Now that it's ready, let's add it to the appropriate trigger point in CloudBolt.

In your CloudBolt instance, navigate to Admin > Actions > Orchestration Actions and click “Provision Server” on the left tab bar. Find the “Post-Provision” trigger point at the bottom of the page and click the “Add an Action” button.

Select “CloudBolt Plug-in” and in the next dialog, click "Add new cloudbolt plug-in".

Specify a name for our new plug-in (Poll for EC2 Init Complete), select the "Amazon Web Services" resource technology, browse to your script, and click "Create".  Selecting the "Amazon Web Services" resource technology ensures this plug-in only runs against AWS resource handlers that you've defined and not others to which this plug-in is not applicable.

Give it a try

Provision a server to one of your AWS-backed CloudBolt environments. Watching the job progress, you'll see that the job is not marked as complete until the server is fully reachable and SSH access is available.

Questions? Comments? Concerns?

Don't hesitate to reach out to me (rkilcoyne@cloudbolt.io) or any of the CloudBolt Solutions team for help!

Read More

Topics: Automation, AWS, CloudBolt

CloudBolt is Now Free for Lab Use

Posted by Ephraim Baron

10/15/15 12:42 AM

If you’re like most IT shops, you have a place for testing out new products, technologies, and applications.  You may call it your lab, your skunkworks, or your Area 51.  (If you have a different name, share it in a reply to this post.)  Your lab is where you separate truth from hype.  It’s where you make sure things work in your environment, on your equipment, with your team.  CloudBolt’s powerful and intuitive cloud management platform makes a great addition to your lab.  Keep reading to find out why.  Or to get started right away, just

Download the OVA

Gathering Clouds

Using multiple cloud providers is easy with CloudBoltPerhaps you’re just getting started with cloud computing, or maybe you’re testing out multiple clouds, public and private.  Either way, CloudBolt’s powerful and intuitive cloud management platform can help.  CloudBolt serves as a manager-of-managers to provide single-pane-of-glass visibility and control.  We integrate a wide range of virtualization, cloud, automation, and orchestration tools and technologies.

Our cloud management platform lets you test a variety of configurations on the back-end while providing a consistent end-user front-end.  This can be very handy for vendor bakeoffs.  For example, you can spin up an Apache Tomcat environment in your own data center as well as in a public cloud provider based in, say, Singapore and compare provisioning time, system performance, and user experience.

Because CloudBolt coordinates each cloud integration, you don’t have to keep up with each vendor’s terminology, APIs, and quirks.  You just set up the initial integration for each provider and then lather, rinse, and repeat.  Instead of becoming locked into a particular vendor’s offerings, you can select the best environment for each workload.  We even discover and integrate with your existing deployments, and we stay in sync regardless of whether you make changes using CloudBolt or through vendor-specific tools.

Free, as in Beer

CloudBolt is now free for lab use


CloudBolt is pleased to announce that our award winning cloud platform is now available for free for up to 25 VMs in non-production lab environments.  Why?  Because we’re confident in the value of our product.  We use it every day, and it enables us to be highly productive and cost-effective.  We believe it will do the same for you.

To get started, just go to our download page and click the link to 

Download the OVA

Then fill out the license request and we’ll email you a free license that never expires.  System prerequisites and installation instructions are available on our documentation site. When you've finished installing CloudBolt, follow the Getting Started Guide and you’ll be up and running in minutes.  Contact us if you need any assistance.  To jumpstart your CloudBolt deployment, schedule a demo and we’ll walk you through the features that matter most to you.

Read More

Topics: Cloud Management, Licensing, CloudBolt

Network Transformation: Thinking Outside the Middlebox

Posted by Ephraim Baron

9/16/15 12:00 PM

Stuck in the Past

The original computers were relatively dumb, single-purpose machines.  Logic was hard-wired into the circuitry, and the devices were designed for a single task – arithmetic, decryption, tabulation, etc.  Today the idea of having to use separate devices for every application or task would seem ludicrous, yet that’s essentially how networks still operate.

Current networks consist primarily of boxes.  There are routers, and switches, and signaling equipment along with other “middleboxes”, specialized devices with specific functions.  Examples of middleboxes include firewalls, load balancers, proxies, intrusion detection systems, and WAN optimization appliances.  Each of these devices requires configuration and administration.  The result is complexity and inefficiency. 

Middleboxes are highly specialized devices that add complexityImagine a data pipeline of that passes through all these devices.  How many times must a packet be opened, inspected, and acted upon?  How many device operating systems and interfaces do systems administrators need to know/learn?  How many opportunities are there for misconfiguration or device failure?  What happens when rules on different boxes conflict?  Who can troubleshoot traffic that passes through all these devices?

Rewriting the Rules

The goal of Software Defined Networking (SDN) is to abstract network functions from dependency on hardware.  This, in turn, enables high-level programmability of the network.  In technical jargon, SDN involves separating the data plane (the devices directly involved in moving packets) from the control plane (the logic of how traffic gets from source to destination).  Rather than focusing on boxes, SDN starts with data flows, which are the building blocks of higher-level network logic and functionality.

In a software defined networking world, physical middleboxes become software programs.  This is known as Network Functions Virtualization (NFV), and it allows for functional abstraction along with a common management framework.  By virtualizing network functions in the control plane, network design can be policy-based rather than device-based.  In many ways this is analogous to other types of software development.  The developer writes code that specifies the input, output, and transformations.  Compilers and optimizers determine the most efficient machine instructions and execution logic to implement the code.

As alluded to above, networks built from boxes are complex and difficult to troubleshoot.  I’ll illustrate with an anecdote.  I used to work for a large software company that ran many online services.  Over time, our core routers became cluttered from several generations of configurations and access control lists (ACLs).  It got to the point where our network team had almost no idea which rules were still active.  Their answer – which is probably familiar to anyone who has managed a legacy network – was to sequentially turn rules off to see which applications broke.

Interconnecting multiple network devices can be messyBy contrast, SDN promises a global network view.  Functional complexity is managed at the network edge, while the core primarily focuses on moving packets.  As Jennifer Rexford, one of the pioneers in the field of SDN, sees it, “SDN allows for network-wide visibility and control, which we've never had before.”

From Theory to Practice

Assuming the ideas discussed up to this point are of interest to you, how do you get started?  The answer is “it depends”.  As with many technologies, what is and what is not SDN can be confusing.  Some companies define SDN as a way to automate and configure their physical switches and routers.  This is the approach many network hardware vendors have taken, and it makes sense in terms of their core competencies and existing customer base.  Cisco, for example, continues to add SDN capabilities to its Nexus line of switches.

Many software vendors, on the other hand, define SDN in terms of network virtualization.  With this approach, the goal is to provide an abstraction layer that essentially commoditizes the hardware components.  This follows the same path that server virtualization has taken.  Virtualization leader VMware’s acquisition of SDN pioneer Nicira in 2012 made sense within the context of their strategy to deliver the Software Defined Data Center (SDDC).  Within a year of the acquisition, VMware had combined elements of Nicira technology with its well-established network virtual switching to form its NSX product family

The addition of NSX to VMware’s vSphere product line means that users and administrators can create, modify, and administer networks as easily as they can manage virtual machines (VMs) – without adding a lot of expensive network gear.  The combination of VMware server virtualization and NSX network virtualization enables IT organizations to deliver services with the same agility and flexibility as the large public cloud providers.

CloudBolt is pleased to announce extensive integration of VMware NSX technology in our 5.2 product release.  CloudBolt enables enterprise IT to operate as a cloud service provider.  Our powerfully simple cloud management platform integrates on-premises resources with public clouds, automation scripting tools, and domain-specific technologies.  By delivering a responsive and agile alternative to shadow IT, CloudBolt gives users what they want, when they want it.

To learn more, contact us to schedule a demo.
Or if you prefer to jump right in,
Download CloudBolt

Read More

Topics: Network Virtualization, SDN, VMware NSX, CloudBolt