It seems to me that the world of IT infrastructure had a big shift several years ago when Amazon Web Services (AWS) and Microsoft Azure started offering off-premise infrastructure, which was collectively called “the cloud”. This is the concept of having your “servers” be hosted virtually, off-site, in the “cloud”, which has many, many benefits – with cost-savings being a big one.
However, it wasn’t until recently that I think there has been another wave, and a confluence of several technologies which has led to a new era for infrastructure and systems management. It seems like the next plateau for the industry is going to be a few things:
- Docker: The capability to completely “containerize” your application, using Docker for example, and deploy it to any Docker host. This is where the application and all of it’s dependencies can be packaged-up and run on anything that supports Docker, making the “operating system” irrelevant. Most technologies, including ASP.NET are now pretty much ready for this. In fact, I wrote a blog post about that, here.
- Infrastructure as Code (IaC): The capability to dynamically provision the infrastructure your app needs. Using so-called “infrastructure as code”, the development area defines the servers, load-balancing, etc that is needed for the application in a JSON configuration file, and that environment is created, or updated upon every deploy. This means that your configuration for v1.4.2 of your application is stored in source control, right along with the app. Need to revert to v1.4.2, and the deploy process will bring the hardware back to that configuration for that version too. I have an upcoming blog post on using Puppet for this.
- On-Prem “cloud services” (a.k.a private cloud): most people agree that AWS and Azure are really great ways to manage infrastructure. However, they cost money for every bit of usage. “Cloud is the new mainframe”, as I always say, because you can’t make a move without there being charge-back! Most companies have infrastructure already on-site or simply need to host systems on-premise. So, how could you take the convenience of “managing your infrastructure in the cloud”, but have the physical hardware on-prem? There seem to be two significant offerings for this at the moment: Microsoft Azure Stack (also see here), which brings the actual Azure console to your on-prem hardware, and OpenStack, which is an open-source cloud management software – which is what this blog post is about.
To my brain, it seems like these three techniques is where infrastructure is heading because it supports: legacy apps who just want to deploy to VM “servers” like always. It supports simple/modern apps who want to containerize and easily scale up and out, using something like Docker. And it would support complex apps that can perhaps containerize, but also have complex “server” setups, and could manage that with infrastructure-as-code (IaC). For example, you have message queues, middle-tier “processing” servers, reporting servers, etc. which all need to be set up a specific way and/or verified whenever you deploy your application.
Put another way, now that virtualization has been mastered, and now that containerization has been mastered, the last pieces that are needed is to: make as much of this self-service, as is possible and then automate all the things!!
With that said, here’s a look at what I’ve learned from playing around with OpenStack.
OpenStack was created by Rackspace and NASA in the year 2010. It was meant to be an open source way that you could offer a “cloud” experience using your own hardware, on-premise. Specifically, offering Infrastructure as a Service (IaaS).
Here is a great video I found which explains the origin story pretty well:
Looks pretty cool, right?!
Getting the software:
Believe it or not, this is where it starts getting complexticated. I spent quite a bit of time in the documentation, and from what I gather, it seems like there are three options:
- Download and run “devstack”, a complete, single-server edition which has all of the features – which I’ll cover here.
- Use one of the MANY customized versions, for many different distributions of Linux, see here: https://www.openstack.org/marketplace/distros/
- Use one of the many hosting providers who happen to use OpenStack, in case you don’t want to use Amazon, Microsoft, IBM, or Google. See here: https://www.openstack.org/marketplace/
In my case, since most larger companies use Red Hat Enterprise Linux (RHEL), and with CentOS being the civilian version of the same, I went with that approach. I downloaded and installed CentOS on a bare-metal machine. This is not a small nor simple “program” to install, this is meant to be the core of your data center, so digging to the real, multi-server install will be beyond the scope of this post.
Installing the software:
I just followed the instructions on this page: http://docs.openstack.org/developer/devstack/ for CentOS/RHEL. I guess there are many thoughts on WHERE to put this source code. Since this is for the entire server, I am putting the source in:
Then, per the documentation, a new user called “stack” is created, and has “sudo” permission. So, while still logged in as root, I change the owner of that path:
# chown stack:stack -R /opt/openstack/
this means change the owner to user “stack” and group “stack”, recursively for all subdirectories and files starting from /opt/openstack/. This is needed because stack.sh needs to be run as the stack user, and that stack user needs to have permission to that path. With that done, I went into the devstack folder and ran: stack.sh. This runs for a while – like, several minutes.
When complete, it gives you a status like this:
Great, I’ll just open a browser and navigate to http://192.168.1.24/dashboard and I’ll be off and running!
Configuring the firewall:
Well wait, the first thing I found was that despite what the output of stack.sh said, I couldn’t get to the website. Well, CentOS and RHEL ship with SELinux and with the firewall turned-on, by default. So, we need to allow those two ports in the firewall. First, we edit the iptables:
$ sudo nano /etc/sysconfig/iptables
and then we add two lines, after the existing “ACCEPT” entries, but before any of the “REJECT” entries:
-A INPUT -p tcp -m state –state NEW -m tcp –dport 80 -j ACCEPT
-A INPUT -p tcp -m state –state NEW -m tcp –dport 5000 -j ACCEPT
then, exit and save changes. To put these rules in-effect, run:
$ sudo service iptables restart
Now, you should be able to get to the main website.
I can now log into the portal, which is http://192.168.1.24/dashboard for me – but will be different for you, and I see a login screen:
from the output from stack.sh, I see can either log in as admin/nomoresecret or demo/nomoresecret. I’ll log in as admin first.
from the admin side, you can set up all of the things you want to make available to end-users, like VM images, block storage, virtual LAN’s, etc.
Then, if I log in as the demo user, I see a slightly different interface:
and very similar to AWS, I can click on “Instances” and go spin up a new virtual machine:
and as an admin, you can predefine virtual machine configurations, similar to Azure and AWS:
and when done, you can see your new virtual machines:
this worked earlier, but when I went to get a screenshot of it now, it’s in an error state – I keep getting a “block device mapping” error for some reason. But you get the idea. You can normally click on the instance name and even open an SSH prompt from there too.
From reading about this particular product, it seems like this could be a pretty good alternative to manually managing many, many Hyper-V or VMWare servers. In theory, you have OpenStack be your hypervisor and you could literally manage an entire data center of virtual machines from one console. Better, you can define tenants and let development areas provision whatever they need, directly. Even better than that, there are API’s for everything you see, which means that infrastructure provisioning could be automated too, where needed.
For me personally, I homelab with two “servers” that host many virtual machines. In my free time I’d like to see if I can ultimately get OpenStack to replace that, and manage the hardware of those two host machines, via an interface like that. If/when I transition to that, I’ll likely do a blog post or two on it. From reading up on it, it doesn’t seem like a trivial thing – but I do like the idea!