how change my policy of scheduling in hadoop?

I want to change policy of scheduling in Hadoop, how to I can change job order in map reduce automatically.

asked Apr 13, 2016 by ali

1 Answer

Assume you are using Hadoop 2 / YARN.

The configuration parameter named yarn.resourcemanager.scheduler.class controls the class to be used as the resource scheduler for YARN/Hadoop.

The default value for the scheduler class (check more at http://ask.systutorials.com/749/hadoop-2-yarn-default-configuration-values) is:

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

which is the Capacity Scheduler.

To change the scheduler used, you need to assign the scheduler class in yarn-site.xml in the etc/hadoop/ directory under Hadoop installation. Taking the fair scheduler shipped with Hadoop as an example, we can add to yarn-site.xml:

<property>
  <name>yarn.resourcemanager.scheduler.class</name>
  <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>

Each scheduler may have its own options for configurations, such as capacity scheduler and fair scheduler. You may check their manuals to configure and tune them.

answered May 5, 2016 by Eric Z Ma (44,280 points)
edited May 5, 2016 by Eric Z Ma

Please log in or register to answer this question.

Copyright © SysTutorials. User contributions licensed under cc-wiki with attribution required.
Hosted on Dreamhost

...