How to install Hortonworks HDP 2.2 on ubuntu 12.04Friday, Dec 5th, 2014
Hortonworks has recently released HDP 2.2. In this step-by-step guide, we show you how to install HDP 2.2 on a small cluster using Ansible, Cobbler and Ambari. Read this PDF document for full documentation. We installed HDP 2.2 on this 6-node cluster provided by BigBoards:
— Daan Gerits (@daangerits) November 8, 2014
Why all these different tools?
Ambari is a tool for 'provisioning, managing, and monitoring Apache Hadoop clusters'. It is shipped with HDP 2.2. So why do we need the other tools, Cobbler and Ansible? The answer is simple: Automation
Cobbler is a "installation server". That means, with one press of a button, you can tell cobbler to reformat and reinstall a computer. This is especially useful if you have clusters of computers. It takes some time to set it all up. You need to configure PXE, TFTP and DHCP, before you can even configure Cobbler itself. But once, it's running, it saves you so much time. I can now get a clean linux install of 1 or 100 machines in a few minutes. If you're running in the cloud, it's less useful. But for bare metal machines, this is great. Setting up Cobbler is food for a blog in itself, so we won't dive deeper into it here.
Then why Ansible? Ansible provides IT automation, like Chef or Puppet. When Cobbler is done, you have a naked linux install. You need to do some initial setup, like passwordless private-key SSH, before Ambari wants to talk to you. I like Ansible because it only needs SSH to talk to all computers. Again, completely configuring Ansible would take us too far. But here's the gist to set up Ambari:
It assumes you have a group 'cluster' and a group 'cluster-manager' defined in your hosts. For me, that looked like this:
We're ready to install HDP 2.2
If you've done all this, manually or scripted, you are ready to install HDP 2.2 by using Ambari. Just go to your manager node with the browser. For me this was http://nuc01:8080/ and log in with admin:admin.
All done. Time to play.
This is what Ambari looks like when HDP 2.2 is running:
Sadly, hue is not supported when running on ubuntu. That is really weird. But it's not strictly necessary. Also, I didn't see any option to install Spark. You'll have to do that yourself.
All-in-all it was quite easy to set up Hortonworks HDP 2.2 on a Cluster. Ambari looks a lot like Cloudera Manager, although the charts on Ambari don't look as nice. We'll learn more as we actually start playing with the tools in the coming days.