CloudBolt Software Logo

CloudBolt Blog

Network Transformation: Thinking Outside the Middlebox

Posted by Ephraim Baron

9/16/15 12:00 PM

Stuck in the Past

The original computers were relatively dumb, single-purpose machines.  Logic was hard-wired into the circuitry, and the devices were designed for a single task – arithmetic, decryption, tabulation, etc.  Today the idea of having to use separate devices for every application or task would seem ludicrous, yet that’s essentially how networks still operate.

Current networks consist primarily of boxes.  There are routers, and switches, and signaling equipment along with other “middleboxes”, specialized devices with specific functions.  Examples of middleboxes include firewalls, load balancers, proxies, intrusion detection systems, and WAN optimization appliances.  Each of these devices requires configuration and administration.  The result is complexity and inefficiency. 

Middleboxes are highly specialized devices that add complexityImagine a data pipeline of that passes through all these devices.  How many times must a packet be opened, inspected, and acted upon?  How many device operating systems and interfaces do systems administrators need to know/learn?  How many opportunities are there for misconfiguration or device failure?  What happens when rules on different boxes conflict?  Who can troubleshoot traffic that passes through all these devices?

Rewriting the Rules

The goal of Software Defined Networking (SDN) is to abstract network functions from dependency on hardware.  This, in turn, enables high-level programmability of the network.  In technical jargon, SDN involves separating the data plane (the devices directly involved in moving packets) from the control plane (the logic of how traffic gets from source to destination).  Rather than focusing on boxes, SDN starts with data flows, which are the building blocks of higher-level network logic and functionality.

In a software defined networking world, physical middleboxes become software programs.  This is known as Network Functions Virtualization (NFV), and it allows for functional abstraction along with a common management framework.  By virtualizing network functions in the control plane, network design can be policy-based rather than device-based.  In many ways this is analogous to other types of software development.  The developer writes code that specifies the input, output, and transformations.  Compilers and optimizers determine the most efficient machine instructions and execution logic to implement the code.

As alluded to above, networks built from boxes are complex and difficult to troubleshoot.  I’ll illustrate with an anecdote.  I used to work for a large software company that ran many online services.  Over time, our core routers became cluttered from several generations of configurations and access control lists (ACLs).  It got to the point where our network team had almost no idea which rules were still active.  Their answer – which is probably familiar to anyone who has managed a legacy network – was to sequentially turn rules off to see which applications broke.

Interconnecting multiple network devices can be messyBy contrast, SDN promises a global network view.  Functional complexity is managed at the network edge, while the core primarily focuses on moving packets.  As Jennifer Rexford, one of the pioneers in the field of SDN, sees it, “SDN allows for network-wide visibility and control, which we've never had before.”

From Theory to Practice

Assuming the ideas discussed up to this point are of interest to you, how do you get started?  The answer is “it depends”.  As with many technologies, what is and what is not SDN can be confusing.  Some companies define SDN as a way to automate and configure their physical switches and routers.  This is the approach many network hardware vendors have taken, and it makes sense in terms of their core competencies and existing customer base.  Cisco, for example, continues to add SDN capabilities to its Nexus line of switches.

Many software vendors, on the other hand, define SDN in terms of network virtualization.  With this approach, the goal is to provide an abstraction layer that essentially commoditizes the hardware components.  This follows the same path that server virtualization has taken.  Virtualization leader VMware’s acquisition of SDN pioneer Nicira in 2012 made sense within the context of their strategy to deliver the Software Defined Data Center (SDDC).  Within a year of the acquisition, VMware had combined elements of Nicira technology with its well-established network virtual switching to form its NSX product family

The addition of NSX to VMware’s vSphere product line means that users and administrators can create, modify, and administer networks as easily as they can manage virtual machines (VMs) – without adding a lot of expensive network gear.  The combination of VMware server virtualization and NSX network virtualization enables IT organizations to deliver services with the same agility and flexibility as the large public cloud providers.

CloudBolt is pleased to announce extensive integration of VMware NSX technology in our 5.2 product release.  CloudBolt enables enterprise IT to operate as a cloud service provider.  Our powerfully simple cloud management platform integrates on-premises resources with public clouds, automation scripting tools, and domain-specific technologies.  By delivering a responsive and agile alternative to shadow IT, CloudBolt gives users what they want, when they want it.

To learn more, contact us to schedule a demo.
Or if you prefer to jump right in,
Download CloudBolt

Read More

Topics: Network Virtualization, SDN, VMware NSX, CloudBolt

Build private cloud on top of virtualized network

Posted by Justin Nemmers

3/18/13 11:40 AM

Let’s face it. Networks are a pain to implement, maintain, and debug. Additionally, they’re often viewed as fragile enough that many teams generally wish to avoid routinely poking at them by messing with configurations or frequently creating/deleting VLANs.

Implementing a flexible and scalable private cloud environment on an inflexible network will only serve to reduce the flexibility and scalability of a private cloud environment that needs to grow.  In addition, ongoing management of these environments can quickly become difficult when administrators don’t have the ability to easily restrict network access by group, or have the ability to rapidly create new stand-alone networks for a specific application, group, or requirement.

virtualized networking separates logical from physical
Separate the logical from the phisical network.  Network virtualization does for networks what server virtualziation did for servers. You can't talk virtualization management without also talking about network virtualization management.

Enter network virtualization!  When implemented in your environment, and made consumable by a Cloud Manager, network virtualization suddenly breaks the network stack wide open.  In fact, I’d argue that until you virtualize the network, even private cloud alone is only partly useful.  Why?  Well, for several reasons:

  • Private clouds alone are limited by their ability to meet capacity demands. 
  • Eventually, that private cloud will run out of data center space, or will need to otherwise expand out of it’s shell. 
  • Whether your private cloud is fully on-prem, or you’re using a virtual private cloud model from someone like Amazon Web Services (AWS), the inflexibility of unifying that networking layer can be a difficult hurdle to surmount. 

Let’s expand on this AWS example.  Amazon offers a Virtual Private Cloud (VPC) that is essentially a private cloud hosted in the public cloud. Confused yet?  Don’t be. AWS uses advanced network and security parameters to effectively cordon off your cloud-based VMs from other tenants, allowing for secure communication and private networking in your hosted private cloud. They do this by manipulating the network layers in the hypervisors. AWS’ use of networking, although advanced, has its limitations, though. For instance, although VPCs can span availability zones, separate regions may require separate VPC definitions, leaving the networking integration to the user. In those cases, your local facility will have to implement it’s own routes to properly send traffic to the correct VPC. Although you can certainly work through those limitations, a hosted private cloud like that is wholly dependent on AWS. 

It doesn’t get any easier when your private cloud is completely on-prem. Be it demand growth, or a shift in requirements or priorities, networking is likely to be one of the significant bottlenecks in the growth and success of your private cloud.  

This is why a technology like network virtualization is so important. Implementing network virtualization in a private cloud environment (be it greenfield, or layered into an existing brownfield environment) allows you to approach new requirements with flexibility in mind and little concern over the networking infrastructure. Just make sure that your underlying network has the Layer 2 capacity for required traffic, and then start to build your environment above that.

In order to attain the flexibility of network virtualization on top of your private cloud, you need effective management. This goes beyond creating a handful of networks and handing them over to users.  Understanding what networks are required by which users and groups, and then ensuring that access is properly controlled is more than critical: it’s a requirement that must be met, or the network will remain a significant impedance to growth. Especially when it is time to expand the reach of your private cloud—whether that be adding capacity, layering in additional technologies, or perhaps looking to securely and safely make use of public cloud resources (congrats, you now have a hybrid cloud!)—Management of the entire stack is an imperative part of the solution. Deploy applications, resources, and networks all in one pass, no matter the environment. That’s the promise of network virtualization. CloudBolt makes it usable.

Read More

Topics: Network Virtualization, Software Defined Network, Management, Implementation, AWS

Cloud Managers Will Change IT Forever

Posted by John Menkart

2/20/13 10:37 AM

In numerous conversations with customers and analysts it has become clear that a consensus across the industry is that Cloud Managers are as game changing for IT as server and network virtualization themselves.  Among those looking longer term at the promise of Cloud Computing (Public, Private and Hybrid), it is clear that the Cloud Manager will become the keystone of value.  Many people’s opinion is that Cloud Managers are the initiator of next major wave of change in IT environments.  How?  Well let’s look to the past to predict the future.

Proprietary Everything

Back in the early 80’s, general purpose computers were first spreading across the business environment. These systems were in the form of fully-proprietary Mainframes, Minicomputers. The hardware (CPU, Memory, Storage, etc.), Operating Systems and any even any available software were all from the specific computer manufacturer (vendors included DEC, Prime, Harris, IBM, HP, DG amongst others).  Businesses couldn’t even acquire compilers for their systems from a third party.  They were only available from the system’s manufacturer.

Commodity OS leads to Commodity Hardware

Agility and maturity of IT environments step 1

The advent of broad interest and adoption of Unix started a sea change in the IT world.  As more hardware vendors supported Unix it became easier to migrate from one vendor’s system to another.  Additionally, vendors began building their systems based on commodity x86-compatible microprocessors as opposed to building proprietary CPU architectures optimized around their proprietary OS.

Architecture-compatible hardware not only accelerated the move to commodity OS (Unix, Linux and Windows), but in turn, increased pressure on vendors to fully commoditize server hardware.  The resulting commoditization of hardware systems steeply drove down prices.  To this day, server hardware largely remains a commodity.

Virtualization Commoditizes Servers

 

Agility and maturity of IT environments step 2

Despite less expensive commodity operating systems and commodity hardware, modernizing enterprise IT organizations were still spending large sums on new server hardware in order to accommodate the rapidly growing demand of new applications.  In large part, IT organizations had a problem taking full advantage of the hardware resources they are spending on.  Server utilization become a real issue.  Procurement of servers still took a considerable amount of time due to organizational processes.  Every new server required a significant amount of effort to purchase, rack and stack, and eventually deploy.  Power and cooling requirements became a significant concern.  The integration of storage, networking, and software deployment and maintenance still caused considerable delays into workflows that are reliant on new hardware systems.

Server virtualization arrives commercially in the late 1990’s and starts getting considerable traction in the mid 2000’s.  Virtualization of the underlying physical hardware provides an answer to the thorny utilization issue by enabling multiple individual server workloads that have low individual utilization to be consolidated on a single physical server.  Virtualization also provides a limited  solution for the  the procurement problem, and helps with the power and cooling issues posed by rampant hardware server growth. Areas of networking, storage, and application management remain disjointed, and typically still require similar times to effectively implement as before the advent of virtualization thus becoming a major impediment to flexibility in the enterprise IT shops.

Now we find ourselves in 2013.  Most enterprise IT shops have implemented some level of virtualization. All of the SaaS and Cloud-based service providers have standardized on virtualization. Virtual servers can be created rapidly and at no perceived cost other than associated licenses, so VM Servers are essentially a commodity, although the market share for the underlying (enabling) technology is clearly in VMware’s favor at this point.

The problem with these commodity VM servers is that making them fully available for use still hinges on integrating them with other parts of the IT environment that are far from commodity and complex to configure.  The VM’s dependency on network, automation tools, storage, etc. hinder the speed and flexibility of the IT group to configure and provide rapid access to these resources for the business.

Network Virtualization arrives

A huge pain point in flexibly deploying applications and workloads is the result of networking technology still being largely based on the physical configuration of network hardware devices across the enterprise. The typical enterprise network is both complex and fragile, which is a condition that dos not encourage rapid change in the network layer to accommodate business or mission application requirements. An inflexible network which is available is always preferred to a network that failed because of unintended consequences of a configuration change.

In much the same way as Server Virtualization abstracted the server from the underlying hardware, Network virtualization completely abstracts the logical network from the physical network.  Using network virtualization it is now possible to free the network configuration from the physical devices, enabling rapid deployment of new, and more efficient management of existing virtual networks.  Rapid adoption of network virtualization technology in the future is all but guaranteed.

Commoditizing all IT resources and compute

 

Agility and maturity of IT environments step 3

With both network and server virtualization, we are closer than ever to the real benefit of 'Cloud Computing': the promise of  fully commoditized IT resources and compute.  To get there, however, we need to coordinate and abstract the management and control the modern enterprises’ internal IT resources and compute resources being consumed in external public cloud providers.

To enable rapid and flexible coordination of the IT resources, the management of those enterprise application resources must be abstracted from the underlying tools.  The specific technologies (server virt, network virt, automation, storage, public cloud provider, etc.) involved are viewed as commodity, and can be exchanged or deprecated without negatively affecting the business capabilities of the enterprise IT. Additionally this abstraction allows the IT organization to flexibly adopt new and emerging technologies to add functionality and capability without exposing the business to the often sharp edges leading edge technology.

The necessary resource abstraction and control is the domain of the not just the virtualization manager-- but really the Cloud Manager. In short, the Cloud Manager commoditizes compute by commoditizing the IT resources across the enterprise and beyond.

With such an important role it is no wonder that every vendor wants to pitch a solution in this space. The orientation or bias of the various vendors’ approaches in developing a Cloud Manager for enterprise IT will play a critical role in the ultimate success of the products and customers that implement them.

Read More

Topics: Network Virtualization, IT Challenges, Virtualization, Cloud Manager, John, Enterprise, IT Organization, Agility, Compute, Hardware

CloudBolt Releases C2 v3.6.0

Posted by Justin Nemmers

2/14/13 3:31 PM

We're happy to announce the release of CloudBolt C2 v3.6.0!

Building on the ground-breaking Network Virtualization capabilites we released in v3.5.0, we've created the ability to directly manage network virtualization-provided layer 3 networking (i.e. routing) directly from C2.

C2 also now supports KVM-QEMU, further expanding the supported virtualization platforms that it centrally manages.

We've also added many more visual cues throught the user interface. You'll now see appropriate vendor icons for items including resource handlers like VMware vSphere and vCenter, AWS and QEMU, as well as Operating Systems, Configuration Management systems (Puppet, Chef, HP Server Automation), and Network Virtualization (Nicira by VMware).

Do you have a large number of users, but don't want to connect C2 to LDAP or Active Directory? Not a problem anymore, as C2 can now import users from a csv file.

From the beginning, CloudBolt enables plain old virtualzation environments to provide resources in a Cloud-ified manner: virtualization becomes Infrastructure as a Service and Platfform as a Service.  Strating with v3.6.0, we enable users to request multiple servers from multiple environments.  Previously, they could request multiple servers, but only from on envronment at a time.

We've also invested a bunch of time in performance tuning the UI, including adding capabilities to filter the server list by the OS family a server belongs to.

C2 can also now query a Configuration Management system to determine which virtual machines in your environment are also managed by a supported CM system so that C2 can enable more fine-grained application and life-cycle management of those VMs.

Ready to upgrade?  Hit up our support portal (login required) for details.  Want to kick the tires?  Request a download now!

Download C2

Read More

Topics: Nicira, Network Virtualization, Feature, Management, Virtualization, VMware, Cloud Manager, Upgrade, Release Notes

CloudBolt C2 & VMware (Nicira) network virt: Why it's a big deal

Posted by Bernard Sanders

1/2/13 1:33 PM

In an enterprise organization without virtualization technology, the creation and configuration of new networks requires a network engineering team to manually configure network devices. The process is often difficult and arduous as a lengthy troubleshooting process ensues between network engineers and the server team when the network does not function as expected.  The advent of server virtualization has added the virtualization administrator to this process to make the new network available within the hypervisor. The process is sufficiently cumbersome enough that IT administrators aggressively avoid the activity, and instead reuse and overload existing networks to the detriment of the end users of IT. 

Avoiding the creation of new networks impacts development and quality assurance more adversely than production as it causes pre-production environments to drift from production. That creates unexpected environmental problems when applications are promoted to production (e.g., hours after the site goes down, “ah, it doesn’t work when service X is not on the same subnet!”).

Messy Network No Virtualization

The level of difficulty of a technical procedure should not dictate the processes undertaken. Technology should act as a catalyst of change, rather than an inhibitor.  VMware network virtualization (The bits they purchased from Nicira, which was formerly NVP, or Network Virtualization Platform) moves the state of technology forward by abstracting the network from the underlying hardware, and eliminates dependency on high-end networking hardware and specialists trained in the configuration of proprietary hardware. 

This advancement provides an opportunity and a challenge: 

  • The opportunity is to move the task of configuring networking closer to the groups that need the networking. 
  • The challenge is to expose this functionality in a way that is simple enough that the average consumer of IT can take advantage of it. 

CloudBolt C2 solves this challenge for an array of technology, including network virtualization, and is currently the only CMP with network virtualization management capabilities.

Combined with network virtualization, CloudBolt C2 provides an exceedingly simple web interface that IT consumers can use to request new virtual networks as well as servers.  C2 guides these requests through an approval process and then takes action on them, taking care of communication with more complex back-end systems like VMware network virtualization, server virtualization, and configuration management systems. The level of integration between C2 and network virtualization is unequaled in the industry. The CloudBolt C2 Enterprise edition enables features such as adding fine-grained permission controls around the creation and deletion of virtual networks, granting end users the ability to save a composite network-server order as an “application” for rapid re-deployment of complex services, and gathering network utilization data from Nicira to incorporate into C2’s cost-tracking system.

The end result is that users can easily create entire labs and data center environments in minutes with just a few graphically-driven choices.  Not only does this accelerate and automate existing tasks facing IT organization, it also enables them to work in ways that were heretofore impractical or impossible, granting end users an unsurpassed level of self-sufficiency.

 

Read More

Topics: Nicira, Network Virtualization, Software Defined Network, Consumability, Network