Modifier and Type | Field and Description |
---|---|
static String |
ALLOCATION_FILE |
protected static String |
ALLOW_UNDECLARED_POOLS
Whether pools can be created that were not specified in the FS configuration file
|
static String |
ASSIGN_MULTIPLE
Whether to assign multiple containers in one check-in.
|
protected static String |
CONTINUOUS_SCHEDULING_ENABLED
Deprecated.
Continuous scheduling is known to cause locking issue inside
the scheduler in larger cluster, more than 100 nodes, use
ASSIGN_MULTIPLE to improve container allocation ramp up. |
protected static String |
CONTINUOUS_SCHEDULING_SLEEP_MS
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
protected static String |
DEFAULT_ALLOCATION_FILE |
protected static boolean |
DEFAULT_ALLOW_UNDECLARED_POOLS |
protected static boolean |
DEFAULT_ASSIGN_MULTIPLE |
protected static boolean |
DEFAULT_CONTINUOUS_SCHEDULING_ENABLED
Deprecated.
|
protected static int |
DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS
Deprecated.
|
protected static long |
DEFAULT_LOCALITY_DELAY_NODE_MS
Deprecated.
|
protected static long |
DEFAULT_LOCALITY_DELAY_RACK_MS
Deprecated.
|
protected static float |
DEFAULT_LOCALITY_THRESHOLD |
protected static float |
DEFAULT_LOCALITY_THRESHOLD_NODE |
protected static float |
DEFAULT_LOCALITY_THRESHOLD_RACK |
protected static int |
DEFAULT_MAX_ASSIGN |
protected static boolean |
DEFAULT_PREEMPTION |
protected static float |
DEFAULT_PREEMPTION_THRESHOLD |
static int |
DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB
Deprecated.
|
static int |
DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
Deprecated.
|
static float |
DEFAULT_RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE |
protected static boolean |
DEFAULT_SIZE_BASED_WEIGHT |
static int |
DEFAULT_UPDATE_INTERVAL_MS |
protected static boolean |
DEFAULT_USER_AS_DEFAULT_QUEUE |
protected static int |
DEFAULT_WAIT_TIME_BEFORE_KILL |
protected static long |
DEFAULT_WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS |
static String |
DYNAMIC_MAX_ASSIGN
Maximum number of containers to assign on each check-in.
|
protected static String |
LOCALITY_DELAY_NODE_MS
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
protected static String |
LOCALITY_DELAY_RACK_MS
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
protected static String |
LOCALITY_THRESHOLD_NODE
Cluster threshold for node locality.
|
protected static String |
LOCALITY_THRESHOLD_RACK
Cluster threshold for rack locality.
|
static org.slf4j.Logger |
LOG |
protected static String |
MAX_ASSIGN
Specify exact number of containers to assign on each heartbeat, if dynamic
max assign is turned off.
|
protected static String |
PREEMPTION
Whether preemption is enabled.
|
protected static String |
PREEMPTION_THRESHOLD |
static String |
RESERVABLE_NODES
Ratio of nodes available for an app to make an reservation on.
|
static float |
RESERVABLE_NODES_DEFAULT |
static String |
RM_SCHEDULER_INCREMENT_ALLOCATION_MB
Deprecated.
The preferred way to configure the increment is by using the
yarn.resource-types.{RESOURCE_NAME}.increment-allocation property,
for memory: yarn.resource-types.memory-mb.increment-allocation
|
static String |
RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
Deprecated.
The preferred way to configure the increment is by using the
yarn.resource-types.{RESOURCE_NAME}.increment-allocation property,
for CPU: yarn.resource-types.vcores.increment-allocation
|
static String |
RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE
Threshold for container size for making a container reservation as a
multiple of increment allocation.
|
protected static String |
SIZE_BASED_WEIGHT
Whether to give more weight to apps requiring many resources.
|
static String |
UPDATE_INTERVAL_MS
The update interval for calculating resources in FairScheduler .
|
protected static String |
USER_AS_DEFAULT_QUEUE
Whether to use the user name as the queue name (instead of "default") if
the request does not specify a queue.
|
protected static String |
WAIT_TIME_BEFORE_KILL |
protected static String |
WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS
Configurable delay (ms) before an app's starvation is considered after
it is identified.
|
Constructor and Description |
---|
FairSchedulerConfiguration() |
FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf) |
Modifier and Type | Method and Description |
---|---|
boolean |
getAssignMultiple() |
int |
getContinuousSchedulingSleepMs()
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
org.apache.hadoop.yarn.api.records.Resource |
getIncrementAllocation() |
long |
getLocalityDelayNodeMs()
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
long |
getLocalityDelayRackMs()
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
float |
getLocalityThresholdNode() |
float |
getLocalityThresholdRack() |
int |
getMaxAssign() |
org.apache.hadoop.yarn.api.records.Resource |
getMaximumAllocation() |
org.apache.hadoop.yarn.api.records.Resource |
getMinimumAllocation() |
boolean |
getPreemptionEnabled() |
float |
getPreemptionUtilizationThreshold() |
float |
getReservableNodes() |
float |
getReservationThresholdIncrementMultiple() |
boolean |
getSizeBasedWeight() |
long |
getUpdateInterval() |
boolean |
getUsePortForNodeName() |
int |
getWaitTimeBeforeKill() |
long |
getWaitTimeBeforeNextStarvationCheck() |
boolean |
isContinuousSchedulingEnabled()
Deprecated.
use
ASSIGN_MULTIPLE to improve container allocation
ramp up. |
boolean |
isMaxAssignDynamic() |
static ConfigurableResource |
parseResourceConfigValue(String value)
Parses a resource config value in one of three forms:
Percentage: "50%" or "40% memory, 60% cpu"
New style resources: "vcores=10, memory-mb=1024"
or "vcores=60%, memory-mb=40%"
Old style resources: "1024 mb, 10 vcores"
In new style resources, any resource that is not specified will be
set to
Long.MAX_VALUE or 100%, as appropriate. |
static ConfigurableResource |
parseResourceConfigValue(String value,
long missing)
Parses a resource config value in one of three forms:
Percentage: "50%" or "40% memory, 60% cpu"
New style resources: "vcores=10, memory-mb=1024"
or "vcores=60%, memory-mb=40%"
Old style resources: "1024 mb, 10 vcores"
In new style resources, any resource that is not specified will be
set to
missing or 0%, as appropriate. |
addDefaultResource, addDeprecation, addDeprecation, addDeprecation, addDeprecation, addDeprecations, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addTags, clear, dumpConfiguration, dumpConfiguration, dumpDeprecatedKeys, get, get, getAllPropertiesByTag, getAllPropertiesByTags, getBoolean, getClass, getClass, getClassByName, getClassByNameOrNull, getClasses, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getDouble, getEnum, getFile, getFinalParameters, getFloat, getInstances, getInt, getInts, getLocalPath, getLong, getLongBytes, getPassword, getPasswordFromConfig, getPasswordFromCredentialProviders, getPattern, getPropertySources, getProps, getPropsWithPrefix, getRange, getRaw, getResource, getSocketAddr, getSocketAddr, getStorageSize, getStorageSize, getStringCollection, getStrings, getStrings, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDurationHelper, getTimeDurations, getTrimmed, getTrimmed, getTrimmedStringCollection, getTrimmedStrings, getTrimmedStrings, getValByRegex, hasWarnedDeprecation, isDeprecated, isPropertyTag, iterator, main, onlyKeyExists, readFields, reloadConfiguration, reloadExistingConfigurations, set, set, setAllowNullValueProperties, setBoolean, setBooleanIfUnset, setClass, setClassLoader, setDeprecatedProperties, setDouble, setEnum, setFloat, setIfUnset, setInt, setLong, setPattern, setQuietMode, setRestrictSystemProperties, setRestrictSystemPropertiesDefault, setRestrictSystemProps, setSocketAddr, setStorageSize, setStrings, setTimeDuration, size, toString, unset, updateConnectAddr, updateConnectAddr, write, writeXml, writeXml, writeXml
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
forEach, spliterator
public static final org.slf4j.Logger LOG
@Deprecated public static final String RM_SCHEDULER_INCREMENT_ALLOCATION_MB
@Deprecated public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB
@Deprecated public static final String RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
@Deprecated public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
public static final String RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE
public static final float DEFAULT_RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE
public static final String ALLOCATION_FILE
protected static final String DEFAULT_ALLOCATION_FILE
protected static final String ALLOW_UNDECLARED_POOLS
protected static final boolean DEFAULT_ALLOW_UNDECLARED_POOLS
protected static final String USER_AS_DEFAULT_QUEUE
protected static final boolean DEFAULT_USER_AS_DEFAULT_QUEUE
protected static final float DEFAULT_LOCALITY_THRESHOLD
protected static final String LOCALITY_THRESHOLD_NODE
protected static final float DEFAULT_LOCALITY_THRESHOLD_NODE
protected static final String LOCALITY_THRESHOLD_RACK
protected static final float DEFAULT_LOCALITY_THRESHOLD_RACK
@Deprecated protected static final String LOCALITY_DELAY_NODE_MS
CONTINUOUS_SCHEDULING_ENABLED
is enabled@Deprecated protected static final long DEFAULT_LOCALITY_DELAY_NODE_MS
@Deprecated protected static final String LOCALITY_DELAY_RACK_MS
CONTINUOUS_SCHEDULING_ENABLED
is enabled@Deprecated protected static final long DEFAULT_LOCALITY_DELAY_RACK_MS
@Deprecated protected static final String CONTINUOUS_SCHEDULING_ENABLED
ASSIGN_MULTIPLE
to improve container allocation ramp up.@Deprecated protected static final boolean DEFAULT_CONTINUOUS_SCHEDULING_ENABLED
@Deprecated protected static final String CONTINUOUS_SCHEDULING_SLEEP_MS
CONTINUOUS_SCHEDULING_ENABLED
is enabled@Deprecated protected static final int DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS
protected static final String PREEMPTION
protected static final boolean DEFAULT_PREEMPTION
protected static final String PREEMPTION_THRESHOLD
protected static final float DEFAULT_PREEMPTION_THRESHOLD
protected static final String WAIT_TIME_BEFORE_KILL
protected static final int DEFAULT_WAIT_TIME_BEFORE_KILL
protected static final String WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS
WAIT_TIME_BEFORE_KILL
and enough heartbeats.
This is intended to be a backdoor on production clusters, and hence
intentionally not documented.protected static final long DEFAULT_WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS
public static final String ASSIGN_MULTIPLE
protected static final boolean DEFAULT_ASSIGN_MULTIPLE
protected static final String SIZE_BASED_WEIGHT
protected static final boolean DEFAULT_SIZE_BASED_WEIGHT
public static final String DYNAMIC_MAX_ASSIGN
protected static final String MAX_ASSIGN
protected static final int DEFAULT_MAX_ASSIGN
public static final String UPDATE_INTERVAL_MS
public static final int DEFAULT_UPDATE_INTERVAL_MS
public static final String RESERVABLE_NODES
public static final float RESERVABLE_NODES_DEFAULT
public FairSchedulerConfiguration()
public FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf)
public org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()
public org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()
public org.apache.hadoop.yarn.api.records.Resource getIncrementAllocation()
public float getReservationThresholdIncrementMultiple()
public float getLocalityThresholdNode()
public float getLocalityThresholdRack()
@Deprecated public boolean isContinuousSchedulingEnabled()
ASSIGN_MULTIPLE
to improve container allocation
ramp up.@Deprecated public int getContinuousSchedulingSleepMs()
CONTINUOUS_SCHEDULING_ENABLED
deprecation@Deprecated public long getLocalityDelayNodeMs()
CONTINUOUS_SCHEDULING_ENABLED
deprecation@Deprecated public long getLocalityDelayRackMs()
CONTINUOUS_SCHEDULING_ENABLED
deprecationpublic boolean getPreemptionEnabled()
public float getPreemptionUtilizationThreshold()
public boolean getAssignMultiple()
public boolean isMaxAssignDynamic()
public int getMaxAssign()
public boolean getSizeBasedWeight()
public long getWaitTimeBeforeNextStarvationCheck()
public int getWaitTimeBeforeKill()
public boolean getUsePortForNodeName()
public float getReservableNodes()
public static ConfigurableResource parseResourceConfigValue(String value) throws AllocationConfigurationException
Long.MAX_VALUE
or 100%, as appropriate. Also, in the new
style resources, units are not allowed. Units are assumed from the resource
manager's settings for the resources when the value isn't a percentage.value
- the resource definition to parseConfigurableResource
that represents the parsed valueAllocationConfigurationException
- if the raw value is not a valid
resource definitionpublic static ConfigurableResource parseResourceConfigValue(String value, long missing) throws AllocationConfigurationException
missing
or 0%, as appropriate. Also, in the new style
resources, units are not allowed. Units are assumed from the resource
manager's settings for the resources when the value isn't a percentage.
The missing
parameter is only used in the case of new style
resources without percentages. With new style resources with percentages,
any missing resources will be assumed to be 100% because percentages are
only used with maximum resource limits.value
- the resource definition to parsemissing
- the value to use for any unspecified resourcesConfigurableResource
that represents the parsed valueAllocationConfigurationException
- if the raw value is not a valid
resource definitionpublic long getUpdateInterval()
Copyright © 2008–2021 Apache Software Foundation. All rights reserved.