Ask Your Question
0

Sahara: Change Java heap space

asked 2014-07-24 11:07:48 -0600

datts gravatar image

Hi everybody,

I have created a cluster hadoop (When create Node group template, I have modified mapred.child.java.opts: "-Xmx200m" to "-Xmx1024m"). Then i ssh to master-instance and run: "hadoop jar hadoop-examples-1.2.1.jar pi 10 100", I get the error:

java.lang.OutOfMemoryError: Java heap space

Check file /etc/hadoop/hadoop-env.sh. I see:

Default: export HADOOP_CLIENT_OPTS="-Xmx128m $HADOOP_CLIENT_OPTS"

Then, I edit 128 to 1024:

export HADOOP_CLIENT_OPTS="-Xmx1024m $HADOOP_CLIENT_OPTS"

When run "hadoop jar hadoop-examples-1.2.1.jar pi 10 100", it's worked! But, my cluster still in a state: "Watting" and ip_master:50030/jobtracker.jps not running?

Why HADOOP_CLIENT_OPTS in /etc/hadoop/hadoop-env.sh not automatic update when create cluster?

What is the problem here? Thank you very much!

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
2

answered 2014-08-08 13:12:12 -0600

alazarev gravatar image

First of all, which process failed with OutOfMemoryError? I would say that it is not likely something controlled by client opts.

Second, mapred.child.java.opts and HADOOP_CLIENT_OPTS control the same params, but in different ways. Overwriting of mapred.child.java.opts will lead to new value in mapred-site.xml and I believe you have right value because of "I have modified mapred.child.java.opts". This has nothing to do with hadoop-env.sh file.

Third, Sahara doesn't expect you do any job executions until cluster is in Active state. Cluster in "Waiting" state means that Sahara didn't finish hadoop installation and cluster is not ready for job executions yet. You need to fix this issue first.

edit flag offensive delete link more

Comments

Hi alazarew,

This issue related to my questions at: https://ask.openstack.org/en/question...

I'm still trying to fix this error :(

P/s: The time now, i'm installing OpenStack Icehouse on KVM virtual machine (Ubuntu12.04). I suspect the problem here. I will reinstall it on a physical server and report results.

Thanks.

datts gravatar imagedatts ( 2014-08-11 09:30:25 -0600 )edit

OpenStack works well on virtual machine. Virtualization may have impact on performance (nested virtualization helps a lot here), but in general there should be no difference.

alazarev gravatar imagealazarev ( 2014-08-11 15:49:34 -0600 )edit
datts gravatar imagedatts ( 2014-08-13 02:49:37 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-07-24 11:07:48 -0600

Seen: 986 times

Last updated: Jul 24 '14