ow2-proactive / nodesource-addons

ActiveEon ProActive Workflows and Scheduling

How to build

1. Import Gradle project

2. Build shadowJar with ./gradlew shadowJar

How to install

For example for the AWS EC2 infrastructure:

1. Copy .jar file from infrastructures/infrastructure-aws-ec2/build/libs into ProACtive Workflows and Scheduling Addons folder
2. Add 'org.ow2.proactive.resourcemanager.nodesource.infrastructure.AWSEC2Infrastructure' to [ProActive_Home]config/rm/nodesource/infrastructures

nodesource-addons's Issues

Inject script on boot (if available) for multiple instances

The issue is about script injection on boot and multiple instance creatio and is partially related to the issue ow2-proactive/connector-iaas#47.
On Azure & OpenStack cloud, the acquireNode script can be injected on VM's boot.
In addition, when we have multiple instances to start, some optimizations are made inside connector-iaas' createInstance method to start multiple VMs/instances as fast as possible using SDK features.
The problem is that the scripts need to be customized with instanceTag/id and (according to the model issue of the connector-iaas project) this will force instances to start sequentially.
We thus need to find a way to customize the script for each instance to take advantage of clouds SDK's parallel VMs deployment features.

Improve description of infrastructures parameters

For the optional infrastructures parameters which have a default value, it should be presented in the description.

Form submission improvement

Add notifications with appropriate error names during form submission, there is currently no feedback on what is happening behind (infrastructure registration, instances creation and script injection).

All cloud infrastructures should have in their form a field to upload the user's credentials

Currently, the rm.cred that is used by the node to register to the RM is located in the node.jar. If ever the credentials change on the server side, then it will never be possible for the node to connect. To prevent this problem, the best way is to ask the user to provide its own credentials, so we need to add this in the forms of all cloud infrastructures, like for the LocalInfrastructure.

Azure Infrastructure not deploying

Description

• Azure nodes don't get deployed. When using the right credentials, the logs show a successful deployment, but the node is not even shown as "deploying".

• The VM, however, gets created and is accessible through ssh.

• Another issue which might be consequence to this one, is that the VM is not deleted upon undeployment. The VM and all its related resources are left there, incurring costs.

• Upon accessing the deployed VM we can see that node.jar is correctly downloaded, but the launching nohup command fails as java is not installed in a generic machine:

root@azuretest:/var/lib/waagent/custom-script/download/0# cat stderr 2019-05-22 13:34:24 URL:http://40.89.153.167:8080/rest/node.jar [132930488/132930488] -> "node.jar" [1] nohup: failed to run command 'java': No such file or directory
root@azuretest:/var/lib/waagent/custom-script/download/0# cat stdout script-ID 3fa86921-4cc4-4457-aa4c-776ff5e93bf9

To investigate/fix:

• Can we deploy a node with a VM image with java pre-installed?
• Verify that the protocol (PNP, PAMR) and URL parameters are correct.
• Add java installation to the launching script, to avoid using a custom image.

Scheduler.log:

o.o.p.r.c.RMCore] Node source azuretest has been successfully edited o.o.p.r.c.RMCore] Deploy node source azuretest requested by admin o.p.r.n.i.AzureInfrastructure] Validating parameters : [HIDDEN] o.o.p.r.n.NodeSource] [azuretest] Activating the policy Static Policy user access type [ALL], provider access type [ME] o.o.p.r.n.p.NodeSourcePolicy] [azuretest] Acquiring all nodes .r.n.i.ConnectorIaasController] Creating Azure infrastructure : {"credentials":HIDDEN,"id":"azuretest","type":"azure","toBeRemovedOnShutdown":false} .r.n.i.ConnectorIaasController] Azure infrastructure created .m.a.a.AuthenticationAuthority] [Correlation ID: XXXX-XXXX-XXXX] Instance discovery was successful .r.n.i.ConnectorIaasController] Total existing Instances By Infrastructure Id : 0 .r.n.i.ConnectorIaasController] InstanceJson : {"image":"proactive-scaleset","number":"1","credentials":HIDDEN .r.n.i.ConnectorIaasController] Instances ids created : [62038d76-e67a-4951-879c-f64e3b3f98a4] .o.p.r.n.i.AzureInfrastructure] Instances ids created or retrieved : [62038d76-e67a-4951-879c-f64e3b3f98a4] .r.n.i.ConnectorIaasController] Trying to execute script for instance id:62038d76-e67a-4951-879c-f64e3b3f98a4 .r.n.i.ConnectorIaasController] Executed successfully script for instance id:62038d76-e67a-4951-879c-f64e3b3f98a4 .r.n.i.ConnectorIaasController] InstanceScriptJson: {"scripts":["/bin/bash -c 'echo script-ID; echo 3fa86921-4cc4-4457-aa4c-776ff5e93bf9; wget -nv http://40.89.153.167:8080/rest/node.jar; nohup java -jar node.jar -Dproactive.pamr.router.address=40.89.153.167 -DinstanceId=62038d76-e67a-4951-879c-f64e3b3f98a4 -Dproactive.useIPaddress=true -r pnp://server-test:64738/ -s azuretest -w 1 &'"]} .r.n.i.ConnectorIaasController] Script result: [{"instanceId":"62038d76-e67a-4951-879c-f64e3b3f98a4","output":"","error":""}] o.o.p.r.n.NodeSource] [azuretest] Pinging alive nodes : 0 o.o.p.r.c.RMCore] Node source azuretest has been successfully deployed

JMX RMI nodes service address uses the private IP when deployed in AWS

The JMX RMI service address shouldn't use the private IP of the instance in AWS.

This prevent the monitoring to work.

In order to use the public IP of the AWS image the following property needs to be set when starting the node :
-Dproactive.hostname=xxx

Code improvements

Some code improvements need to be done in the entire project according to the reviews on PR #31:

• better parameters management (using a HashMap instead of Object[]?)
• code refactoring (can be partially based on the most recent Microsoft Azure addon)
• clean unit tests

Azure: enable PNP communications

In Azure, by default, there is no domain name associated with the public IP address attached to the VMs we deployed. As a consequence, a VM only has the knowledge of its private IP address even if there is a public IP attached to it (along with open ports in the security group), so PNP will only listen on the local/private IP address which is unreachable from an external scheduler.
The problem can be solved either by generating and adding a domain name to the instance (which can be done programmatically using the SDK), or by extending the injection script to retrieve the public IP of the VM and add it as an option to the node.jar execution.