org.ow2.proactive.resourcemanager.nodesource.infrastructure

Class InfrastructureManager

java.lang.Object

org.ow2.proactive.resourcemanager.nodesource.infrastructure.InfrastructureManager

All Implemented Interfaces:

Serializable, NodeSourcePlugin

Direct Known Subclasses:

BatchJobInfrastructure, DefaultInfrastructureManager, HostsFileBasedInfrastructureManager, LocalInfrastructure

public abstract class InfrastructureManager
extends Object
implements NodeSourcePlugin

Represents underlying infrastructure and defines ways to acquire / release nodes.
Acquisition requests are supposed to be asynchronous. When new new node is available it should register itself into the resource manager by calling internalRegisterAcquiredNode(Node)
To define a new infrastructure manager - define a way to add information about further node acquisition implementing configure(Object...) - define a way to acquire a single node from underlying infrastructure in acquireNode() - define a way to acquire all available nodes from the infrastructure in the method acquireAllNodes() - register available nodes in the resource manager using internalRegisterAcquiredNode(Node), so they till be taken into account. - add the name of new class to the resource manager configuration file (config/rm/nodesource/infrastructures).

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static interface	InfrastructureManager.PersistedInfraVariablesHandler<T> Interface that allows handling the runtime variables through a functional point of view.
static class	InfrastructureManager.RMDeployingNodeAccessor Helper nested class.

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	ELASTIC
protected static org.apache.log4j.Logger	logger class' logger
protected NodeSource	nodeSource manager's node source
protected Map<String,Serializable>	persistedInfraVariables Store information about the running infrastructure.
protected Lock	readLock Use this lock to wrap a read access to runtime variables
protected static String	RM_URL_KEY
protected Lock	writeLock Use this lock to wrap a write access to runtime variables

Constructor Summary

Constructors

Constructor and Description
InfrastructureManager()

Method Summary

Modifier and Type	Method and Description
abstract void	acquireAllNodes() Asynchronous request of all nodes acquisition.
void	acquireAllNodes(Map<String,?> nodeConfiguration)
abstract void	acquireNode() Asynchronous node acquisition request.
void	acquireNodes(int n, long timeout, Map<String,?> nodeConfiguration) Asynchronous node acquisition request.
void	acquireNodes(int n, Map<String,?> nodeConfiguration) Asynchronous node acquisition request.
protected String	addDeployingNode(String name, String command, String description, long timeout) Creates a new RMDeployingNode's, stores it in a local ArrayList and notify the owning NodeSource of the RMDeployingNode creation
protected RMDeployingNode	addDeployingNodeWithLockAndPersist(String nodeUrl, RMDeployingNode deployingNode)
protected RMDeployingNode	addLostNodeWithLockAndPersist(String nodeUrl, RMDeployingNode lostNode)
protected List<String>	addMultipleDeployingNodes(List<String> nodeNames, String obfuscatedCmd, String message, long nodeTimeout) Add multiple deploying nodes
protected String	buildDeployingNodeURL(String pnName) Builds the name of the deploying node given its name
protected boolean	checkAllNodesAreAcquiredAndDo(List<String> nodeNames, Runnable toRunWhenOK, Runnable toRunWhenKO)
protected boolean	checkNodeIsAcquiredAndDo(String nodeName, Runnable toRunWhenOK, Runnable toRunWhenKO) Check if the Node with the given name is already acquired ( usefull when the launcher of the process which is supposed to launch a RMNode waits for its registration ).
protected abstract void	configure(Object... parameters) Adds information required to deploy nodes in the future.
protected boolean	declareDeployingNodeLost(String toUpdateURL, String description) Declares a deploying node lost.
protected org.ow2.proactive.resourcemanager.utils.CommandLineBuilder	getDefaultCommandLineBuilder(org.ow2.proactive.resourcemanager.utils.OperatingSystem targetOS) This method returns a CommandLineBuilder filled in with "default" settings.
List<RMDeployingNode>	getDeployingAndLostNodes() To retrieve nodes whose registration status is deploying or lost.
protected Collection<RMDeployingNode>	getDeployingNodesWithLock()
RMDeployingNode	getDeployingOrLostNode(String nodeUrl)
abstract String	getDescription()
protected org.ow2.proactive.resourcemanager.utils.CommandLineBuilder	getEmptyCommandLineBuilder() Returns an empty CommandLineBuilder
protected Collection<RMDeployingNode>	getLostNodesWithLock()
Map<String,String>	getMeta()
protected Set<String>	getPersistedDeployingNodesUrl()
protected <T> T	getPersistedInfraVariable(InfrastructureManager.PersistedInfraVariablesHandler<T> t) Acquire the read lock and call the handle method of the handler given in parameter.
protected Set<String>	getPersistedLostNodesUrl()
protected String	getRmUrl()
Map<Integer,String>	getSectionDescriptions()
protected abstract void	initializePersistedInfraVariables() This method should initialize a value in the runtime variables map for all the runtime variables that will be used in the class.
void	internalConfigure(Object... parameters) called by the node source at configuration time.
void	internalNotifyDownNode(String nodeName, String nodeUrl, org.objectweb.proactive.core.node.Node node) This method is called by the system when a Node is detected as DOWN.
RMDeployingNode	internalRegisterAcquiredNode(org.objectweb.proactive.core.node.Node node) This method is called by the RMCore to notify the InfrastructureManager that a new node was added to the core.
boolean	internalRemoveDeployingNode(String pnUrl) To remove a deploying node given its url
void	internalRemoveNode(org.objectweb.proactive.core.node.Node node) Performs some cleanup ( essentially removal of the cached node ) and call removeNodeImpl
void	internalShutDown() Called by the node source at shutdown time.
protected boolean	isShutDown()
boolean	isUsingDeployingNode()
protected void	multipleDeclareDeployingNodeLost(List<String> deployingNodes, String message) Declare multiple nodes lost
protected abstract void	notifyAcquiredNode(org.objectweb.proactive.core.node.Node node) Notifies the implementation of the infrastructure manager that a new node has been registered.
protected void	notifyDeployingNodeLost(String pnURL) Notifies the user that the deploying node was lost or removed (because of a timeout, user interaction...) Default empty implementation is provided because implementors don't necessary use this feature.
abstract void	notifyDownNode(String nodeName, String nodeUrl, org.objectweb.proactive.core.node.Node node) Define what needs to be done in an infrastructure implementation when a node is detected as down.
void	onDownNodeReconnection(org.objectweb.proactive.core.node.Node node) Called by the system every time a node that is DOWN reconnects and changes its status to FREE or BUSY.
void	persistInfrastructureVariables() If nodes recovery is enabled, persist the current state of the node source.
void	reconfigure(Object... updatedInfrastructureParameters) Reconfigure the infrastructure of a potentially already deployed node source with the given parameters.
void	recoverPersistedInfraVariables(Map<String,Serializable> persistedInfrastructureVariables) Copy all the runtime variables map given in parameter to the runtime variables map of this infrastructure.
protected void	removeDownNodePriorToNotify(String nodeName) This method can be overriden by subclasses, especially if they do not want the node to be removed from the InfrastructureManager map when detected as down.
abstract void	removeNode(org.objectweb.proactive.core.node.Node node) Removes the node from the resource manager.
RMNode	searchForNotAcquiredRmNode(String nodeUrl) Look for a RMNode in data structures holding lost and deploying nodes.
void	setNodeSource(NodeSource nodeSource) Sets an infrastructure node source.
protected <T> T	setPersistedInfraVariable(InfrastructureManager.PersistedInfraVariablesHandler<T> t) Acquire the write lock, and then: 1) call the handle method of the handler given in parameter 2) call the method that persist in database the runtime variables.
void	setPersistedNodeSourceData(NodeSourceData nodeSourceData)
protected void	setRmDbManager(RMDBManager dbManager)
protected void	setRmUrl(String rmUrl)
protected void	shutDown() Notify this infrastructure it is going to be shut down along with its nodesource.
RMDeployingNode	update(RMDeployingNode rmNode) Updates a deploying node.
protected boolean	updateDeployingNodeDescription(String toUpdateURL, String newDescription) To update the description of a deploying node.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

class' logger

nodeSource

protected NodeSource nodeSource

manager's node source

ELASTIC

protected static final String ELASTIC

See Also:

Constant Field Values

RM_URL_KEY

protected static final String RM_URL_KEY

See Also:

Constant Field Values

persistedInfraVariables

protected Map<String,Serializable> persistedInfraVariables

Store information about the running infrastructure. The map holds the name of monitored information and its value. The variables stored in this map should allow the full recovery of an infrastructure state. All accesses to this map must be synchronized.

readLock

protected transient Lock readLock

Use this lock to wrap a read access to runtime variables

writeLock

protected transient Lock writeLock

Use this lock to wrap a write access to runtime variables

Constructor Detail

InfrastructureManager

public InfrastructureManager()

Method Detail

getMeta

public Map<String,String> getMeta()

Specified by:

getMeta in interface NodeSourcePlugin

getPersistedInfraVariable

protected <T> T getPersistedInfraVariable(InfrastructureManager.PersistedInfraVariablesHandler<T> t)

Acquire the read lock and call the handle method of the handler given in parameter.

Returns:

the value returned by the handle method

setPersistedInfraVariable

protected <T> T setPersistedInfraVariable(InfrastructureManager.PersistedInfraVariablesHandler<T> t)

Acquire the write lock, and then: 1) call the handle method of the handler given in parameter 2) call the method that persist in database the runtime variables.

Returns:

the return value of the handle method

setNodeSource

public void setNodeSource(NodeSource nodeSource)

Sets an infrastructure node source. Also sets the field rmUrl if it is not provided by users.

Parameters:

nodeSource - policy node source

recoverPersistedInfraVariables

public void recoverPersistedInfraVariables(Map<String,Serializable> persistedInfrastructureVariables)

Copy all the runtime variables map given in parameter to the runtime variables map of this infrastructure.

Parameters:

persistedInfrastructureVariables - the runtime variables to recover.

getDeployingAndLostNodes

public List<RMDeployingNode> getDeployingAndLostNodes()

To retrieve nodes whose registration status is deploying or lost.

Returns:

nodes whose registration status is deploying or lost.

getDeployingOrLostNode

public RMDeployingNode getDeployingOrLostNode(String nodeUrl)

internalRemoveDeployingNode

public final boolean internalRemoveDeployingNode(String pnUrl)

To remove a deploying node given its url

Parameters:

pnUrl - the url of the deploying node to remove.

Returns:

true if successful, false otherwise

internalRemoveNode

public final void internalRemoveNode(org.objectweb.proactive.core.node.Node node)
                              throws RMException

Performs some cleanup ( essentially removal of the cached node ) and call removeNodeImpl

Parameters:

node - the node to be removed

Throws:

RMException

internalNotifyDownNode

public void internalNotifyDownNode(String nodeName,
                                   String nodeUrl,
                                   org.objectweb.proactive.core.node.Node node)
                            throws RMException

This method is called by the system when a Node is detected as DOWN.

Parameters:

node - the ProActive Programming Node that is down. This parameter can be null if no information about the node can be retrieved apart from its url.

nodeUrl - the URL of the node that is down.

Throws:

RMException - if any problems occurred.

removeDownNodePriorToNotify

protected void removeDownNodePriorToNotify(String nodeName)

This method can be overriden by subclasses, especially if they do not want the node to be removed from the InfrastructureManager map when detected as down.

Parameters:

nodeName - the node to remove

notifyDownNode

public abstract void notifyDownNode(String nodeName,
                                    String nodeUrl,
                                    org.objectweb.proactive.core.node.Node node)
                             throws RMException

Define what needs to be done in an infrastructure implementation when a node is detected as down.

Parameters:

node - the ProActive Programming Node that is down.

Throws:

RMException - if any problems occurred.

internalRegisterAcquiredNode

public final RMDeployingNode internalRegisterAcquiredNode(org.objectweb.proactive.core.node.Node node)
                                                   throws RMException

This method is called by the RMCore to notify the InfrastructureManager that a new node was added to the core. If the IM throws an exception, the node is not added. Note that this method is in mutual exclusion with checkNodeIsAcquiredAndDo(String, Runnable, Runnable) At this point, if a previous call to addDeployingNode(String, String, String, long) was made (means that implementor uses deploying nodes features), this method ensures that the implementation method (see notifyAcquiredNode(Node) is only called if no timeout has occurred for the associated deploying node.

Parameters:

node - the newly added node

Throws:

RMException

addMultipleDeployingNodes

protected List<String> addMultipleDeployingNodes(List<String> nodeNames,
                                                 String obfuscatedCmd,
                                                 String message,
                                                 long nodeTimeout)

Add multiple deploying nodes

Parameters:

nodeNames - node names which will be created

obfuscatedCmd - obfuscated command used to start the node

message - user message

nodeTimeout - node timeout used

Returns:

a list of deploying nodes urls

multipleDeclareDeployingNodeLost

protected void multipleDeclareDeployingNodeLost(List<String> deployingNodes,
                                                String message)

Declare multiple nodes lost

Parameters:

deployingNodes - list of urls of deploying nodes

message - user message

internalConfigure

public final void internalConfigure(Object... parameters)

called by the node source at configuration time. Shifts the parameter array once done to let implementation only care about their own configurable parameters. This method acquires the write lock to manipulate runtime variables.

Parameters:

parameters - the parameters of the infrastructure manager

Throws:

IllegalArgumentException - if the parameters are invalid

isShutDown

protected boolean isShutDown()

internalShutDown

public final void internalShutDown()

Called by the node source at shutdown time. First removes every registered Deploying and Lost nodes and delegates the call to the implementation thanks to the method shutDown()

getDescription

public abstract String getDescription()

Returns:

The Description of the implemented Infrastructure

configure

protected abstract void configure(Object... parameters)

Adds information required to deploy nodes in the future. Do not initiate a real nodes deployment/acquisition as it's up to the policy. The runtime variables of an infrastructure should not be explicitly persisted in this method as the link with the node source is not properly set at the time of calling this method. They will be persisted at the end of the initialization in RMCore#createNodeSource

Parameters:

parameters - of the infrastructure manager

Throws:

IllegalArgumentException - if the parameters are invalid

reconfigure

public void reconfigure(Object... updatedInfrastructureParameters)

Reconfigure the infrastructure of a potentially already deployed node source with the given parameters. Implementations are free to handle override of parameters as they wish.

Parameters:

updatedInfrastructureParameters - parameters potentially containing updated dynamic parameters

Throws:

IllegalArgumentException - if parameters are incorrect

See Also:

Configurable.dynamic()

acquireNode

public abstract void acquireNode()

Asynchronous node acquisition request. Proactive node should be registered by calling internalRegisterAcquiredNode(Node)

acquireNodes

public void acquireNodes(int n,
                         Map<String,?> nodeConfiguration)

Asynchronous node acquisition request. The request may be skipped if another concurrent request is on-going.

Parameters:

n - the number of nodes to acquire

nodeConfiguration - the configuration for nodes to be acquired

acquireNodes

public void acquireNodes(int n,
                         long timeout,
                         Map<String,?> nodeConfiguration)

Asynchronous node acquisition request.

Parameters:

n - the number of nodes to acquire

timeout - The maxime waiting time (in milliseconds) before starting to acquire nodes This timeout is used to acquire locks for concurrent deployment if needed. The request will be skipped if it can't be started within the specified timeout.

nodeConfiguration - the configuration for nodes to be acquired

acquireAllNodes

public abstract void acquireAllNodes()

Asynchronous request of all nodes acquisition. Proactive nodes should be registered by calling internalRegisterAcquiredNode(Node)

acquireAllNodes

public void acquireAllNodes(Map<String,?> nodeConfiguration)

removeNode

public abstract void removeNode(org.objectweb.proactive.core.node.Node node)
                         throws RMException

Removes the node from the resource manager.

Parameters:

node - the node to release.

Throws:

RMException - if any problems occurred.

notifyDeployingNodeLost

protected void notifyDeployingNodeLost(String pnURL)

Notifies the user that the deploying node was lost or removed (because of a timeout, user interaction...) Default empty implementation is provided because implementors don't necessary use this feature. Anyway, if they decide to do so, they can override this method, for instance, to change a flag that would get a control loop to exit...

Parameters:

pnURL - the deploying node's URL for which one the timeout occurred.

notifyAcquiredNode

protected abstract void notifyAcquiredNode(org.objectweb.proactive.core.node.Node node)
                                    throws RMException

Notifies the implementation of the infrastructure manager that a new node has been registered. If this method throws an exception, the node registration will be discarded. This method is always called if implementor doesn't use deploying nodes (no call to addDeployingNode(String, String, String, long) was made), and is called only for deploying nodes for which one no timeout occurred.

Parameters:

node - the newly registered node

Throws:

RMException - if the implementation does not approve the node acquisition request

shutDown

protected void shutDown()

Notify this infrastructure it is going to be shut down along with its nodesource. All necessary cleanup should be done here.

persistInfrastructureVariables

public void persistInfrastructureVariables()

If nodes recovery is enabled, persist the current state of the node source. If the node source data of this infrastructure is not know, it is first fetched in database. The node source data is never persisted if the current infrastructure is the RMConstants.DEFAULT_LOCAL_NODES_NODE_SOURCE_NAME.

setRmDbManager

protected void setRmDbManager(RMDBManager dbManager)

getDefaultCommandLineBuilder

protected final org.ow2.proactive.resourcemanager.utils.CommandLineBuilder getDefaultCommandLineBuilder(org.ow2.proactive.resourcemanager.utils.OperatingSystem targetOS)

This method returns a CommandLineBuilder filled in with "default" settings. That means that the returned CommandLineBuilder is useable as such.

• It tries to set the Java Path to use, either JAVA_HOME retrieved from your environment or java.home set by Java itself.
• The target operating system is set to OperatingSystem.UNIX
• If a ProActive configuration file is provided, it is used as such.
• Finally, it tries to set the nodesource's name, the rm's URL and the node's name.

Parameters:

targetOS - the operating system on which one the node will be deployed

getEmptyCommandLineBuilder

protected final org.ow2.proactive.resourcemanager.utils.CommandLineBuilder getEmptyCommandLineBuilder()

Returns an empty CommandLineBuilder

Returns:

Returns an empty CommandLineBuilder

addDeployingNode

protected final String addDeployingNode(String name,
                                        String command,
                                        String description,
                                        long timeout)

Creates a new RMDeployingNode's, stores it in a local ArrayList and notify the owning NodeSource of the RMDeployingNode creation

Parameters:

name - The RMDeployingNode's name.

description - The RMDeployingNode's description

timeout - after which one the deploying node will be declared lost. ( node acquisition after this timeout is discarded )

Returns:

The newly created RMDeployingNode's URL.

Throws:

UnsupportedOperationException - if the infrastructure manager is shuting down

updateDeployingNodeDescription

protected final boolean updateDeployingNodeDescription(String toUpdateURL,
                                                       String newDescription)

To update the description of a deploying node. If a timeout has occurred for this node, the update is discarded.

Parameters:

toUpdateURL - The RMDeployingNode's URL whose description will be updated.

newDescription - The new description

Returns:

true in case of success, false if the deploying node is not managed by the IM anymore.

declareDeployingNodeLost

protected final boolean declareDeployingNodeLost(String toUpdateURL,
                                                 String description)

Declares a deploying node lost. Future attempts to modify the deploying node will be ignored.

Parameters:

toUpdateURL - The RMDeployingNode's URL which is to be declared as lost

description - the new rmdeployingnode's description, can be null.

Returns:

true if the method ran successfully, false otherwise.

checkNodeIsAcquiredAndDo

protected final boolean checkNodeIsAcquiredAndDo(String nodeName,
                                                 Runnable toRunWhenOK,
                                                 Runnable toRunWhenKO)

Check if the Node with the given name is already acquired ( usefull when the launcher of the process which is supposed to launch a RMNode waits for its registration ). At the same time, this methode executes the first runnable is the node has been found amoung acquired nodes and executes the second runnable if the node has not been found. Note that this method is in mutual exclusion with internalRegisterAcquiredNode(Node) to avoid races.

Parameters:

nodeName - the node's name

toRunWhenOK - the Runnable that will be launched if the node is already acquired (can be null)

toRunWhenKO - the Runnable that will be launched if the node has not been found among nodesource.getAliveNodes(). (can be null)

Returns:

true if the node with such name is already acquired false otherwise.

checkAllNodesAreAcquiredAndDo

protected final boolean checkAllNodesAreAcquiredAndDo(List<String> nodeNames,
                                                      Runnable toRunWhenOK,
                                                      Runnable toRunWhenKO)

buildDeployingNodeURL

protected String buildDeployingNodeURL(String pnName)

Builds the name of the deploying node given its name

Parameters:

pnName - The name of the deploying node

Returns:

the URL of the deploying node

searchForNotAcquiredRmNode

public RMNode searchForNotAcquiredRmNode(String nodeUrl)

Look for a RMNode in data structures holding lost and deploying nodes. It does not look into the acquired nodes because this data structure holds instances of Node and thus has no information about the acquired RMNode.

Parameters:

nodeUrl -

Returns:

a lost or deploying RMNode that could be found using the given URL, or null

onDownNodeReconnection

public void onDownNodeReconnection(org.objectweb.proactive.core.node.Node node)

Called by the system every time a node that is DOWN reconnects and changes its status to FREE or BUSY.

Parameters:

node - the node that has reconnected.

update

public RMDeployingNode update(RMDeployingNode rmNode)

Updates a deploying node.

The update is performed on deploying nodes first. If no node if found, lost nodes are considered.

Parameters:

rmNode - the new deploying node instance to use.

Returns:

the previous value or null.

initializePersistedInfraVariables

protected abstract void initializePersistedInfraVariables()

This method should initialize a value in the runtime variables map for all the runtime variables that will be used in the class. It is called at initialization time of the infrastructure, just before the configure(Object...) method. This method runs with the write lock acquired.

setPersistedNodeSourceData

public void setPersistedNodeSourceData(NodeSourceData nodeSourceData)

getPersistedDeployingNodesUrl

protected Set<String> getPersistedDeployingNodesUrl()

addDeployingNodeWithLockAndPersist

protected RMDeployingNode addDeployingNodeWithLockAndPersist(String nodeUrl,
                                                             RMDeployingNode deployingNode)

getDeployingNodesWithLock

protected Collection<RMDeployingNode> getDeployingNodesWithLock()

getPersistedLostNodesUrl

protected Set<String> getPersistedLostNodesUrl()

addLostNodeWithLockAndPersist

protected RMDeployingNode addLostNodeWithLockAndPersist(String nodeUrl,
                                                        RMDeployingNode lostNode)

getLostNodesWithLock

protected Collection<RMDeployingNode> getLostNodesWithLock()

isUsingDeployingNode

public boolean isUsingDeployingNode()

setRmUrl

protected void setRmUrl(String rmUrl)

getRmUrl

protected String getRmUrl()

getSectionDescriptions

public Map<Integer,String> getSectionDescriptions()

Specified by:

getSectionDescriptions in interface NodeSourcePlugin