Comments on: HBase/Hadoop on Mac OS X (Pseudo-Distributed)

By: patrick

patrick — Sun, 13 May 2012 03:46:01 +0000

hi,
how do you limit the hdfs filesystem format to just a portion of the disk? i don’t want hadoop to eat up all the storage on my imac.

thanks in advance,
patrick

By: Robert J Berger

Robert J Berger — Wed, 07 Mar 2012 10:08:26 +0000

Oh and this tutorial is now pretty out of date. we've switched to using the Cloudera CDH3 distribution. I havent tried to get CDH3 to work on the Mac. You need to do it from their tarballs, though I believe Homebrew now has formula for building Hadoop and HBase on a Mac.

By: Robert J Berger

Robert J Berger — Wed, 07 Mar 2012 10:05:55 +0000

Manish: I’m pretty sure you need zookeeper for any distributed mode (Psuedo or full). The big thing to remember though is HBase is pretty much useless for anything other than very basic API testing/development in Pseudo Distributed mode. And its funky untill you have at least 5 or 6 regionservers. Also if you plan to do any Map/Reduce you really want the parallelism of multiple machines.

HBase only makes sense if you can start at that kind of scale. You can get away with a single node being the zookeeper, hbase master, namenode, secondarynamenode jobtracker, etc and then have the regionservers, hdfs slaves, tasktracker etc on all the other machines. Though you really want to quickly get to the point where the secondarynamenode is on a machine different than the namenode and to have at least 3 zookeeprs. But that is more about robustness and handling some class of failures than basic functionality.

If you only have the scale or ability to use a single machine you should consider some other kind of store.

By: Manish Bhoge

Manish Bhoge — Wed, 07 Mar 2012 09:46:22 +0000

Good tutorial. I am wondering how Hbase works in Pseudo distributed enviornment without doing set up for zookeeper. we have set up HBase on unix machine in stand alone mode. But as far as i know in distributed mode we need to have zookeeper quoram running. Please suggest.

By: iowissen

iowissen — Fri, 24 Feb 2012 07:20:20 +0000

very helpful tutorial! i followed it also successfully on FreeBSD 9.0-RELEASE with diablo-jdk1.6.0 (ported together with openjdk7).

By: sowmya

sowmya — Fri, 10 Feb 2012 05:46:14 +0000

whether text distribution among nodes
is possible in hadoop 1.0.0 version

By: Jignesh

Jignesh — Fri, 21 Oct 2011 15:30:55 +0000

David,
I am having same problem and when I tried to look for log I got following error.

ERROR org.apache.hadoop.hbase.HServerAddress: Could not resolve the DNS name of unknowne4ce8f3898fc
FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: hostname can’t be null

I have asked HBase forum and they said fix your DNS resolving. But when I go home and trying to work it works but not in the office network. Any idea for resolving this.

And off course this is a great tutorial and still not out of date.

By: Anant Jhingran

Anant Jhingran — Mon, 03 Oct 2011 22:50:00 +0000

Thank you, wonderful tutorial!

By: david gerber

david gerber — Fri, 30 Sep 2011 06:11:05 +0000

I decided to run hbase as standalone because I could not get it to work as pseudo distributed. I know you mentioned hbase list and I will definitely look into that but just by chance maybe somebody or yourself have encountered this error.

ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable’s javadoc for more information

this happens when I run the create ‘table_name’,’value’ in the shell

help would be appreciated if you can.

thanks

By: Robert J Berger

Robert J Berger — Fri, 30 Sep 2011 00:18:22 +0000

In reply to david gerber. Best place to ask such questions is the HBase list