ProActive Quickstart Storm

Storm is a free and open source distributed realtime computation sytem. It works with streams of data, equivalent for realtime processing as Hadoop with batch processing. It's used in realtime analytics, online machine learning and more. It's one of the fastest process engine, one million tuples processed per second per node. It relies on a master(Nimbus) /slave architecture.

In this tutorial you will learn how to easily deploy a Storm environment using ProActive Service Automation, then run some computation on it, and finally release this Storm environment.

1 Deploy Storm Instance


  1. Go to ProActive Service Automation

  2. Register with your credentials that you provided

  3. Go to Workflow Execution , Launch a Service.

  4. Select service-automation in the bucket list.

  5. Click on the Storm icon

  6. You can leave the default-values or you can try to play with options :

    INSTANCE_NAME The name of the service to be deployed
    NUMBER_OF_SUPERVISORS Number of supervisors or workers that will be deployed

    You can check the box of advanced variables for more options:

    PROXYFIED True if a proxy is needed to protect the access to the service endpoint.
    ZOOKEEPER_INSTANCE_NAME Instance name of the targeted Zookeeper service .
    ZOOKEEPER_SERVICE_ID Instance ID of the targeted Zookeeper service.

  7. When you have filled the needed fields you can click on submit. Then you should see in the running services area a new line with your Storm instance

    Now you can click to the endpoint adress to access the UI and see that instance is really deployed.

2 Run topology on your Storm instance


Create your first task

  1. Connect to the ProActive Workflow Studio.

  2. Create a new workflow 

#Firstly you need to change your current repository and go to the one with the topology to run
cd /home/activeeon/

#Retrieve the id of nimbus container (deployment relies on Docker)
nimbus_container=$(docker ps -q -af name=nimbus)

#Use the following link to download the storm-starter.jar. It contains a topology called 'exclamation' to be submitted to the deployed Storm cluster:
https://repo1.maven.org/maven2/org/apache/storm/storm-starter/1.2.2/storm-starter-1.2.2.jar


#Copy the topology to nimbus container
docker cp storm-starter-1.2.2.jar $nimbus_container:storm-starter.jar

Create your second task

You need to add a second bash task which depends on the first one 



#retrieve the id of nimbus container,for this you can reuse
nimbus_container=$(docker ps -q -af name=nimbus)

#execute the topology which has been copied
docker exec $nimbus_container /bin/sh -c 'storm jar /storm-starter.jar org.apache.storm.starter.ExclamationTopology stormTutorial'
Now, you can execute the workflow. As container_name, put the same thing as you put in instance_name when deploying Storm. When workflow executed, you can go the scheduler to verify that there is no faulty task, after this verificaton, you can access the StormUI and see that there is a topology running.



You can execute another topology, you just need to change "stormTutorial" with another name in your task.

3 Delete Storm instance


To delete the instance, you simply need to click on the finish button at the end of the line of your running instance in the ProActive Service Automation. Then you can check on the StormUI to see that it is not running anymore.