ProActive AI Orchestration

API documentation

Scheduler REST    Scheduler GraphQL   Scheduler CLI   Scheduler Java   Workflow Creation Java   Python Client   Jupyter Kernel .NET Client

ProActive AI Orchestration (PAIO) is a complete DSML platform (Data Science and Machine Learning) including a ML Studio, AutoML, Data Science Orchestration and MLOps for the deployment, training, execution and scalability of artificial intelligence and machine learning models on any type of infrastructure. Created for data scientists and ML engineers, the solution is simple to use and accelerate the development and deployment of machine learning models.

ProActive AI Orchestration platform provides a rich catalog of generic machine learning tasks that can be connected together to build either basic or advanced machine learning workflows for various use cases such as: fraud detection, text analysis, online offer recommendations, prediction of equipment failures, facial expression analysis, etc. PAIO workflows enable users to manage machine learning pipelines through the different phases of the development lifecycle and allow them to better control tasks parallelization, by running the tasks on resources matching constraints (Multi-CPU, GPU, FPGA, data locality, libraries, etc).

The ProActive AI Orchestration platform is an open source solution, and it can be tested online without installation on our try platforms here.

The MLOps Dashboard is a centralized platform that streamlines the management and monitoring of AI model deployments. It offers a detailed overview of deployment pipelines, real-time performance metrics allowing for efficient oversight of models and servers in production. The dashboard also tracks CPU and GPU resource usage for both individual models and the entire platform, ensuring optimal performance. Additionally, it includes robust functionalities for detecting and managing data drift, helping users maintain the accuracy and reliability of deployed models over time.

Refer to MLOps Dashboard section for more information.

The following terms are used throughout the documentation:

To use PAIO, you need to select the Machine Learning preset as main catalog in the ProActive Studio. This preset contains a set of buckets containing machine learning tasks and workflows that enables you to upload and prepare data, train a model and test it.

To upload data into the Workflow, you need to use a dataset stored in a CSV file.

This task uploads the data into the workflow that we can for model training and testing.

If you want to skip these steps, you can directly use the Load_Boston_Dataset Task by a simple drag and drop.

This step consists of preparing the data for the training and testing of the predictive model. So in this example, we will simply split our datset into two separate datasets: one for training and one for testing.

To do this, we use the Split_Data Task in the machine_learning bucket.

Using PAIO, you can easily create different ML models in a single experiment and compare their results. This type of experimentation helps you find the best solution for your problem. You can also enrich the ai-machine-learning bucket by adding new ML algorithms and publish or customize an existing task according to your requirements as the tasks are open source.

In this step, we will create two different types of models and then compare their scores to decide which algorithm is most suitable to our problem. As the Boston dataset used for this example consists of predicting price of houses (continuous label). As such, we need to deal with a regression predictive problem.

To solve this problem, we have to choose a regression algorithm to train the predictive model. To see the available regression algorithms available on the PAIO, see ML Regression Section in the ai-machine-learning bucket.

For this example, we will use Linear_Regression Task and Support_Vector_Regression Task.

To evaluate the two learned predictive models, we will use the testing data that was separated out by the Split_Data Task to score our trained models. We can then compare the results of the two models to see which generated better results.

Now the workflow is completed, let’s execute it by:

The Distributed_Auto_ML workflow proposes six algorithms for distributed hyperparameters' optimization. The choice of the sampling/search strategy depends strongly on the tackled problem. Distributed_Auto_ML workflow comes with specific pipelines (parallel or sequential) and visualization tools (Visdom or TensorBoard) as described in the subsections below.

Variables:

How to define the search space:

This subsection describes common building blocks to define a search space:

Which tuner algorithm to choose?

The choice of the tuner depends on the following aspects:

In the following, we briefly describe the different tuners proposed by the Distributed_Auto_ML workflow:

Here is a table that summarizes when to use each algorithm.

The following workflows represent some mathematical functions that can be optimized by the Distributed_Auto_ML tuners.

Himmelblau_Function: is a multi-modal function containing four identical local minima. It’s used to test the performance of optimization algorithms. For more info, please click here.

Mathematical Expression

Kursawe_Multiobjective_Function: is a multiobjective function proposed by Frank Kursawe. It has two objectives (f1, f2) to minimize. For more info, please click here.

Mathematical Expression

The following workflows represent some machine learning and deep learning algorithms that can be optimized. These workflows have several common variables as in Distributed_Auto_ML. Some workflows are characterized by few additional variables.

CIFAR_10_Image_Classification: trains a simple deep CNN on the CIFAR10 images dataset using the Keras library.

The following workflows have common variables with the above illustrated workflows.

CIFAR_10_Image_Classification: trains a simple deep CNN on the CIFAR10 images dataset using the Keras library.

CIFAR_100_Image_Classification: trains a simple deep CNN on the CIFAR100 images dataset using the Keras library.

Image_Object_Detection: trains a YOLO model on the coco dataset using PAIO deep learning generic tasks.

Digits_Classification: python script illustrating an example of multiple machine learning models optimization.

Text_Generation: trains a simple Long Short-Term Memory (LSTM) to learn sequences of characters from 'The Alchemist' book. It’s a novel by Brazilian author Paulo Coelho that was first published in 1988.

The following workflows contain a search space containing a set of possible neural networks architectures that can be used by Distributed_Auto_ML to automatically find the best combinations of neural architectures within the search space.

Single_Handwritten_Digit_Classification: trains a simple deep CNN on the MNIST dataset using the PyTorch library. This example allows to search for two types of neural architectures defined in the Handwritten_Digit_Classification_Search_Space.json file.

Multiple_Objective_Handwritten_Digit_Classification: trains a simple deep CNN on the MNIST dataset using the PyTorch library. This example allows optimizing multiple losses, such as accuracy, number of parameters, and memory access cost (MAC) measure.

The following workflows illustrate some examples of multi-node and multi-gpu distributed learning.

TensorFlow_Keras_Multi_Node_Multi_GPU: is a TensorFlow + Keras workflow template for distributed training (multi-node multi-gpu) with AutoML support.

TensorFlow_Keras_Multi_GPU_Horovod: is a Horovod workflow template that support multi-gpu and AutoML.

The following workflows represent python templates that can be used to implement a generic machine learning task.

Python_Task: is a simple Python task template pre-configured to run with Distributed_Auto_ML.

R_Task: is a simple R task template pre-configured to run with Distributed_Auto_ML.

The following workflows represent a client/server templates that can be used to implement a Federated Learning workflow using PyTorch.

PyTorch_FL_Client_Task: is a Federated Learning Client task template using PyTorch.

PyTorch_FL_Server_Task: is a Federated Learning Server task template using PyTorch.

The following workflows represent a client/server templates that can be used to implement a Federated Learning workflow using TensorFlow/Keras.

TensorFlow_FL_Client_Task: is a Federated Learning Client task template using TensorFlow/Keras.

TensorFlow_FL_Server_Task: is a Federated Learning Server task template using TensorFlow/Keras.

The following workflows uses the federated learning to train a deep Convolutional Neural Network (ConvNet/CNN) on the CIFAR10 images dataset using the Flower library.

PyTorch_Federated_Learning_Example: shows an example of Federated Learning workflow using PyTorch.

TensorFlow_Federated_Learning_Example: shows an example of Federated Learning workflow using TensorFlow/Keras.

References:

The Model Servers Monitoring tab focuses on overseeing the health and performance of the model servers or serving infrastructure. It is divided into two main parts.

In the first part, there are 6 main widgets providing general information about the model servers. These widgets offer valuable insights into the overall performance and usage of the serving infrastructure. Here are the widgets included in this tab:

In the second part of the Model Servers Monitoring tab, there is a table listing the model servers along with their respective specific characteristics. The table provides detailed information about each model server instance. Here are the columns representing the characteristics of the model servers:

At the top of this tab, there is a "New Model Server Instance" button that allows users to launch a new model server. When clicked, a window will open, presenting variables that need to be specified by the user. There are two types of variables: General or Advanced. The Table below displays all the variables used to start a new model server.

In the model servers table, when a model server is selected, it shows another subtable that lists all the models stored in the model registry of that specific model server.

This subtable provides additional information about each model. Here are the columns characterizing each model:

This tab and the other two tabs include a "Refresh" button, which, when clicked, refreshes the page and displays the last update datetime.

Additionally, all tabs offer an autorefresh feature that allows users to specify the time period for automatic refreshing. Users can choose from a list of predefined time periods, such as 15 seconds, 30 seconds, 5 minutes, 15 minutes, 30 minutes, 1 hour, etc., to determine how frequently the page should be refreshed automatically.

All the values displayed in the widgets and tables are calculated based on a selected time window. The time window can be chosen from a list of predefined options located at the top of the page, such as "Last 15 minutes," "Last 30 minutes," "Last 1 hour," "Last 24 hours," "Yesterday," "This month," "Previous month," and more. Alternatively, the user can select "Use Calendar," which activates a calendar feature. By choosing this option, the user can manually select the desired "From" and "To" dates, allowing for a custom time window selection.

The second tab of the MLOps monitoring dashboard is dedicated to the Models Resource Usage, providing users with valuable insights into the CPU and GPU resource utilization. This tab is thoughtfully divided into two main parts to ensure a comprehensive understanding of the system’s resource consumption.

The first part features ten main widgets that offer users a wealth of general information about the CPU and GPU usage. These widgets provide real-time metrics, such as average CPU and GPU utilization, memory consumption, etc. By presenting this data in a visually appealing and easily comprehensible format, users can efficiently monitor and evaluate the overall resource consumption of their system. Whether it’s tracking performance trends or identifying potential bottlenecks, these widgets empower users to make informed decisions and optimize their resources effectively.

In the second part of the Model Resources Usage tab, there are several graphs that display time series data for each model server. These graphs provide users with detailed information about various metrics related to CPU and GPU utilization, memory usage, and power consumption. The first two graphs are related to CPU Resources and the rest of the graphs are related to GPU Resources:

The third tab in this dashboard is dedicated to "Dashboard Resource Usage" providing information about the resource consumption of the entire system, it focuses on monitoring the resources utilized by the MLOps infrastructure as a whole. This tab is divided into two main parts:

The first part focuses on providing information about the overall system. This part includes five main metrics:

In the second part of the "Dashboard Resource Usage" tab, there are time series graphs that provide insights into various metrics related to CPU utilization, memory usage, disk memory, and network traffic. The specific graphs in this section include:

The Drift Notification tab enables users to create and manage drift monitoring instances. It allows the user to detect when the deployed models experience data drift changes in the input data distribution that can affect model performance. By configuring drift notifications, users can receive alerts and take proactive measures to maintain model accuracy and reliability.

This page includes a table listing all the drift monitoring instances. Each column in the table provides specific information about the drift monitoring instance.

At the top of this page, there is a "New Drift Monitoring Instance" button that allows users to create a new monitoring instance. When clicked, a window will open, presenting variables that need to be specified by the user.

In case a drift in the input data occurs, a notification is sent to the dedicated channel specified in the NOTIFICATION_CHANNEL variable. This ensures that users are promptly informed about any significant changes in data distribution that may affect model performance. Here is an example of a drift notification from the ProActive Notification Portal:

AutoFeat loads data from external sources. The dataset could be potentially very large. Initially, only the 10 first data rows are displayed.

The Refresh button enables users to see the last updates made on their data.

Whenever AutoFeat loads data from external sources, it also identifies the datatype of each column. AutoFeat does a great job at datatype recognition. Each decision can be overridden manually by the user, if required.

AutoFeat also creates some summary statistics for each column. A table is displaying the missing values, minimum, maximum, mean and zeros for each numerical feature, and the cardinality (category counts) for each categorical feature.

A preview of the data is displayed in the Data Preprocessing as follows.

It is possible to change a column information. These changes can include:

It is also possible to perform the following actions on the dataset:

Once the encoding parameters are set, the user can proceed to display the encoded dataset by clicking on the Preview Encoded Data. He can also check and compare different encoding methods and/or parameters based on the obtained results.

This page displays the data encoding results based on the selected parameters. At this stage, the user can validate the results by clicking on the button Proceed, or erase the encoded dataset by clicking on the button Delete.

The user can also download the results as a csv file by clicking on the Download button.

You can connect different tasks in a single workflow to get the full pipeline from data preprocessing to model training and deployment. Each task will propagate the acquired variables to its children tasks. The following workflow example Vehicle_Type_Using_Model_Explainability uses the Import_Data_And_Automate_Feature_Engineering task to prepare the data. It is available on the machine_learning_workflows bucket.

This workflow predicts vehicle type based on silhouette measurements, and apply ELI5 and Kernel Explainer to understand the model’s global behavior or specific predictions.

Job Analytics Page includes a search window that allows users to search for jobs based on specific criteria (see screenshot below). The job search panel allows selecting multi-value filters for the following job parameters:

More advanced search options (highlighted in advanced search hints) could be used to provide filter values such as wildcards. For example, names that start with a specific string value are selected using value*. Other supported expressions are: *value for Ends with, *value* for Contains, !value for Not equals, and !*value* for Not contains.

Now you can hit the search button to request jobs from the scheduler database according to the provided filter values. The search bar at the top shows a summary of the active search filters.

You can use any jupyter platform. We recommend to use jupyter lab. To launch it from your terminal after having installed it:

or in daemon mode:

When opened, click on the ProActive icon to open a notebook based on the ProActive kernel.

As a quick start, we recommend the user to run the #%help() pragma using the following script:

This script gives a brief description of all the different pragmas that the ProActive Kernel provides.

To get a more detailed description of a needed pragma, the user can run the following script:

Finally, to have the hand on more parameters and features, the user should use ActiveEon Studio portals. The main ones are the Resource Manager, the Scheduling Portal and the Workflow Execution.

The example below shows how the user can directly monitor his submitted job’s execution in the scheduling portal:

To show the resource manager portal related to the host you are connected to, just run:

For the related scheduling portal:

To monitor your jobs with Workflow Execution inside Jupyter, use:

Machine Learning Bucket contains various open source tasks that can be easily used by a simple drag and drop.

It is possible to enrich the ML Bucket by adding your own tasks. (see section 4.3)

It is also possible to customize the code of the generic ML tasks. In this case, you need to drag and drop the targeted task to modify its code in the Task Implementation section.

A fork execution environment is a new Java Virtual Machine (JVM) which is started exclusively to execute a task. Starting a new JVM means that the task inside it will run in a new environment. This environment can be set up by the creator of the task. A new JVMs is set up with a new classpath, new system properties and more customization.

We used a Docker fork environment for all the ML tasks. activeeon/dlm3 was used as a docker container for all tasks. If your task needs to install new ML libraries which are not available in this container, then, use your own docker container or an appropriate environment with the needed libraries.

The Catalog menu provides the possibility for a user to publish newly created or/and update tasks inside Machine Learning Bucket, you need just to click on Catalog Menu then Publish current Workflow to the Catalog. Choose machine-leaning Bucket to store your newly added workflow on it. If the Task with the same name already exists in the 'machine-leaning' bucket, then, it will be updated. We recommend submitting Tasks with a commit message for easier differentiation between the different submitted versions.

The quickstart tutorial on try.activeeon.com shows you how to build a simple workflow using ProActive Studio.

We show below an example of a workflow created with the Studio:

At the left part, are illustrated the General Parameters of the workflow with the following information:

The following workflows present some ML basic examples. These workflows are built using generic ML and data visualization tasks available on the ML and Data Visualization buckets.

Diabetics_Detection_using_K_means: trains and tests a clustering model using Mean_shift algorithm.

Vehicle_Type_Using_Model_Explainability: predicts vehicle type based on silhouette measurements, and apply ELI5 and Kernel Explainer to understand the model’s global behavior or specific predictions.

Parallel_Regression_Model_Training: trains three different regression models.

Parallel_Classification_Model_Training: trains three different classification models.

Nested_Cross_Validation: trains a logistic regression model using a nested cross-validation strategies.

Iris_Flowers_Classification_using_Logistic_Regression: trains and tests a predictive model using logistic_regressive algorithm.

House_Price_Prediction_using_Linear_Regression: trains and tests a regression model using Mean_shift algorithm.

The following workflows present some ML basic examples using AutoML generic tasks available in the ai-machine-learning bucket.

Breast_Cancer_Detection_Using_AutoSklearn_Classifier: tests several ML pipelines and selects the best model for Cancer Breast detection.

California_Housing_Prediction_Using_TPOT_Regressor: tests several ML pipelines and selects the best model for California housing prediction.

The following workflows are designed to detect anomalies in log files. They are constructed using generic tasks which are available on the ai-machine-learning and ai-data-visualization buckets.

Anomaly_Detection_in_Apache_Logs: detects intrusions in apache logs using a predictive model trained using Support Vector Machines algorithm.

Anomaly_detection_in_HDFS_Blocks: trains and test an anomaly detection model for detecting anomalies in HDFS Blocks.

Anomaly_detection_in_HDFS_Nodes: trains and test an anomaly detection model for detecting anomalies in HDFS Nodes.

Unsupervised_Anomaly_Detection: detects anomalies using an Unsupervised One-Class SVM.

The following workflows are designed to feature engineering and fusion.

Data_Fusion_And_Encoding: fuses different data structures.

Data_Anomaly_Detection: detects anomalies on energy consumption by customers.

Diabetics_Results_Visualization_Using_Tableau: visualizes Diabetics Results Using Tableau.

The following workflows are designed for in-memory execution using IPython. IPython enables all types of parallel applications to be developed, executed, debugged, and monitored interactively. For more details, please visit the ipyparallel website.

In_Memory_Iris_Flowers_Classification: classifies Iris flowers using the logistic regression algorithm. This workflow uses an external IPython Engine for in-memory execution.

Start_IPython_Cluster: starts an IPython parallel computing cluster.

The following workflows are designed to train machine learning models on GPU using NVIDIA RAPIDS. This reduces the training time from days to minutes.

Train_Classification_Model_On_GPU: trains a machine learning model for data classification on GPU using NVIDIA RAPIDS.

Train_Multiple_Classification_Models_On_GPU: trains multiple machine learning models for data classification on GPU using NVIDIA RAPIDS.

Train_Multiple_Regression_Models_On_GPU: trains multiple machine learning models for data regression on GPU using NVIDIA RAPIDS.

Train_Regression_Model_On_GPU: trains a machine learning model for data regression on GPU using NVIDIA RAPIDS.

Please find in the table below the list of algorithms which have GPU support and that can be tested using the generic tasks in the ai-machine-learning bucket.

The following workflows present useful examples composed by pre-built Azure cognitive services available on ai-azure-cognitive-services bucket.

Emotion_Detection_in_Bing_News: is a mashup that searches for images of a person using Azure Bing Image Search then performs an emotion detection using Azure Emotion API.

Sentiment_Analysis_in_Bing_News: is a mashup that searches for news related to a given search term using Azure Bing News API then performs a sentiment analysis using Azure Text Analytics API.

The following workflows present useful examples for predictive models training and test using Microsoft Cognitive Toolkit (CNTK).

CNTK_ConvNet: trains a Convolutional Neural Network (CNN) on CIFAR-10 dataset.

CNTK_SimpleNet: trains a 2-layer fully connected deep neural network with 50 hidden dimensions per layer.

GAN_Generate_Fake_MNIST_Images: generates fake MNIST images using a Generative Adversarial Network (GAN).

DCGAN_Generate_Fake_MNIST_Images: generates fake MNIST images using a Deep Convolutional Generative Adversarial Network (DCGAN).

The following workflow presents an example of a workflow built using pre-built Azure cognitive services tasks available on the ai-azure-cognitive-services bucket and custom AI tasks available on the ai-deep-learning bucket.

Custom_Sentiment_Analysis: is a mashup that searches for news related to a given search term using Azure Bing News API then performs a sentiment analysis using a custom deep learning based pretrained model.

This section presents custom AI workflows using tasks available on the ai-deep-learning bucket. Such tasks enable you to train your own AI models by a simple drag and drop of custom AI task.

IMDB_Sentiment_Analysis: trains a model to perform sentiment identification and categorization expressed in a piece of text, especially in order to determine the opinion of IMDB users regarding specific movies [positive or negative]. NOTE: Instead of training a model from scratch, a pre-trained sentiment analysis model is available on this link.

Language_Detection: build an RNN model to perform language detection from a text data.

Train_Image_Classification: trains a model to classify images from ants and bees.

Train_Image_Segmentation: trains a segmentation model using SegNet network on Oxford-IIIT Pet Dataset.

Train_Image_Object_Detection: trains objects using YOLOv3 model on COCO dataset proposed by Microsoft Research.

Deep_Model_Explainability: explains a ResNet-18 model using GradientExplainer.

Search_Train_Image_Classification: queries images from a search engine (Bing or DuckDuckGo) and trains a model to classify them.

This section presents custom AI workflows using tasks available on the ai-deep-learning bucket. Such tasks enable you to test your own AI models by a simple drag and drop of custom AI task.

Image_Classification: predicts a classification model using ResNet_18 network on Ants_vs_Bees Dataset. The pre-trained image classification model is available on this link.

Fake_Celebrity_Faces_Generation: generates a wild diversity of fake faces using a GAN model that was trained based on thousands of real celebrity photos. The pre-trained GAN model is available on this link.

Image_Segmentation: predicts a segmentation model using SegNet network on Oxford-IIIT Pet Dataset. The pre-trained image segmentation model is available on this link.

Image_Object_Detection: detects objects using a pre-trained YOLOv3 model on COCO dataset proposed by Microsoft Research. The pre-trained model is available on this link.

Search_Classify_Images: queries images into search engine (Bing or DuckDuckGo) and predicts a model to classify rocket_vs_plane images. The pre-trained image classiffication model is available on this link.

The following workflows represent python templates that can be used to implement a generic machine learning task.

Horovod_Task: is a template to implement a Horovod task with multi-gpu support.

Horovod_Docker_Task: is a template to implement a Horovod task using a Docker container with multi-gpu support.

Horovod_Slurm_Task: is a template to implement a Horovod task using a native SLURM scheduler with multi-gpu support.

TensorFlow_Task: is a simple TensorFlow task template.

Keras_Task: is a simple Keras task template.

PyTorch_Task: is a simple PyTorch task template.

In the following table, you can find the variables that are common between most of the available AI workflows in PAIO associated to their descriptions.

The ai-machine-learning bucket contains diverse generic ML tasks that enable you to easily compose workflows for predictive models learning and testing. This bucket can be easily customized according to your needs. This bucket offers different options, you can customize it by adding new tasks or update the existing tasks.

The ai-deep-learning bucket contains diverse generic deep learning tasks that enable you to easily compose workflows for predictive models learning and testing. This bucket can be easily customized according to your needs. It offers different options, you can customize it by adding new tasks or update the existing tasks.