How to install Hortonworks HDP 2.2 on ubuntu 12.04

Friday, Dec 5th, 2014

Hortonworks has recently released HDP 2.2. In this step-by-step guide, we show you how to install HDP 2.2 on a small cluster using Ansible, Cobbler and Ambari. Read this PDF document for full documentation. We installed HDP 2.2 on this 6-node cluster provided by BigBoards:

Why all these different tools?

Ambari is a tool for 'provisioning, managing, and monitoring Apache Hadoop clusters'. It is shipped with HDP 2.2. So why do we need the other tools, Cobbler and Ansible? The answer is simple: Automation

Cobbler is a "installation server". That means, with one press of a button, you can tell cobbler to reformat and reinstall a computer. This is especially useful if you have clusters of computers. It takes some time to set it all up. You need to configure PXE, TFTP and DHCP, before you can even configure Cobbler itself. But once, it's running, it saves you so much time. I can now get a clean linux install of 1 or 100 machines in a few minutes. If you're running in the cloud, it's less useful. But for bare metal machines, this is great. Setting up Cobbler is food for a blog in itself, so we won't dive deeper into it here.

Then why Ansible? Ansible provides IT automation, like Chef or Puppet. When Cobbler is done, you have a naked linux install. You need to do some initial setup, like passwordless private-key SSH, before Ambari wants to talk to you. I like Ansible because it only needs SSH to talk to all computers. Again, completely configuring Ansible would take us too far. But here's the gist to set up Ambari:

It assumes you have a group 'cluster' and a group 'cluster-manager' defined in your hosts. For me, that looked like this:

We're ready to install HDP 2.2

If you've done all this, manually or scripted, you are ready to install HDP 2.2 by using Ambari. Just go to your manager node with the browser. For me this was http://nuc01:8080/ and log in with admin:admin.

Next, just follow the Ambari guide:







I decided to put all 'manager' services on nuc01. In that way, all other nodes can be used for real work:



All done. Time to play.

This is what Ambari looks like when HDP 2.2 is running:

Sadly, hue is not supported when running on ubuntu. That is really weird. But it's not strictly necessary. Also, I didn't see any option to install Spark. You'll have to do that yourself.

All-in-all it was quite easy to set up Hortonworks HDP 2.2 on a Cluster. Ambari looks a lot like Cloudera Manager, although the charts on Ambari don't look as nice. We'll learn more as we actually start playing with the tools in the coming days.

Kris
Data architect

Comments

Thanks a lot for sharing this. Very informative. I was wondering whether the same steps can used for Ubuntu 14.0.4.

I'm pretty sure the same applies to Ubuntu 14.04. Definitely share any discrepancies you might discover.

Hi, Did you have any problems on the ambari setup regarding default postgres? I am using an ec2 image of Ubuntu 12.04lts. On setup i use the default settings but i get the following error and ambari is not installed?

Connecting to local database...unable to connect to database
ERROR: psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

ERROR: Exiting with exit code 2.
REASON: Running database init script was failed. psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Add new comment

Image CAPTCHA