Embed your Python model

Suggest edits
Documentation > Embed

Contents

PythonTask syntax 🔗

The PythonTask relies on an underlying ContainerTask but is designed to be transparent and takes only python-related arguments. It takes the following arguments :
  • script String or file, mandatory. The python script to be executed.
  • major Int, optional (default = 3). Major version of python to run - MUST be either 2 or 3.
  • install Sequence of strings, optional (default = empty). The commands to be executed prior to any python packages installation and script execution (to install libraries on the system).
  • libraries Sequence of strings, optional (default = empty). The name of python libraries (through pip) that will be used by the script and need to be installed before (note: as detailed below, installations are only achieved during the first execution of the script, and then stored in a docker image in cache. To force an update, use the forceUpdate argument).
  • forceUpdate Boolean, optional (default = false). Should the libraries installation be forced (to ensure an update for example). If true, the task will perform the installation (and thus the update) even if the library was already installed.

Embedding a Python script 🔗

The toy Python script for this test case is:
import sys

f = open("output.txt", 'w')
f.write(str(arg))

We save this to hello.py. It does nothing but printing its first argument to the file passed as a second argument.

To run this script in OpenMOLE upload hello.py in you workspace. You can then use the following script:
/// Declare the variable
val arg = Val[Int]
val output = Val[File]

// python task
val pythonTask =
  PythonTask(workDirectory / "hello.py") set (
    inputs += arg.mapped,
    outputs += arg,
    outputs += output mapped "output.txt",
  )

val copy = CopyFileHook(output, workDirectory / "result/hello${arg}.txt")
val env = LocalEnvironment(2)

DirectSampling(
  evaluation = pythonTask,
  sampling = arg in (0 to 10)
) hook copy on env

Notions from OpenMOLE are reused in this example. If you're not too familiar with Environments, Groupings, Hooks or Samplings, check the relevant sections of the documentation.

Using Python packages 🔗

One crucial advantage of the Python programming environment is its broad ecosystem of packages, for example used in the machine learning community. You can use python packages in your script, through the libraries argument.

Below is an example, available on the marketplace, which applies a very basic "machine learning" technique (logistic regression) using the scikit-learn python packages, to the outputs of a Netlogo model, providing a sort of "meta-model" to predict the outputs of the simulation model as a function of its parameters without running it.

The syntax for the python task is the following:
val training = Val[File]
val validation = Val[File]
val errdensity = Val[Array[Double]]
val errresistance = Val[Array[Double]]
val score = Val[Double]

val sklearnclassifier =
  PythonTask(
    workDirectory / "logisticregression.py",
    libraries = Seq("pandas","numpy","sklearn")
  ) set (
    inputs += training mapped "data/training.csv",
    inputs += validation mapped "data/validation.csv",
    outputs += errdensity mapped "errdensity",
    outputs += errresistance mapped "errresistance",
    outputs += score mapped "score"
  )

with the following python script:
from sklearn.linear_model import LogisticRegression
import pandas
import numpy

d = pandas.read_csv('data/training.csv')
dp = pandas.read_csv('data/validation.csv')

X = d[['density','resistance']]
y = d['binaryburnt']

Xp = dp[['density','resistance']]
yp = dp['binaryburnt']

clf = LogisticRegression(random_state=0, solver='lbfgs').fit(X, y)
pred = clf.predict(Xp)
prederror = dp.loc[abs(pred - yp)==1]

# define outputs - must be "standard types", not objects (basic types and multidimensional lists)
errdensity = list(prederror['density'])
errresistance = list(prederror['resistance'])
score = clf.score(Xp,yp)

See the market entry for plugging with NetLogo and complete script.