t-wolfeadam / anylogic-pypeline

A custom AnyLogic library for running Python inside an AnyLogic model (Java)

Pypeline

A custom library for AnyLogic that allows you to call Python within a running model. It connects to a local installation of Python, allowing you to make use of any Python library.

It can be used for cases such as:

• Utilizing code that was originally written in Python without having to port to Java
• Writing complex algorithms in Python that you can call in Java, optionally passing objects/data between the languages
• Being able to use any Python-exclusive library
• Using simulation as a testbed for testing trained artificial intelligence policies

Getting Started

These instructions will get Pypeline integrated with AnyLogic. They are described more thoroughly in the provided wiki (which is what the user guide migrated to).

Prerequisites

You need to have AnyLogic with any valid license (PLE, University, or Professional) and **any version of Python 3 - except from the Windows store - installed on your machine.

**Note: Python from the Windows store is not supported due to the inability to externally call its executable. Any other source is supported (e.g., official installer, anaconda, system package managers).

Installation

1. Download the Pypeline.jar file (from the releases) and place it somewhere it won't be moved (or accidentally deleted).
2. Add it to your AnyLogic palette. A step-by-step explanation of how to do this is available in the AnyLogic help article "Managing Libraries".
3. You should see a new palette item for Pypeline with the custom Python Communicator agent.

Quick start guide

This section goes over testing the connection works and a simple tutorial.

For a full explanation of how to use, including a deeper description of the available functions, please refer to the wiki.

Testing connection

To ensure proper connection is made, first run a simple test:

1. Create a new AnyLogic model
2. Drag in a Python Communicator from the Pypeline palette tab; keep its default name ("pyCommunicator")
3. Run the model
4. In your running model, click the Communicator object
5. The inspection window should show the version of Python and the path to the Python executable that's being used

If you do not see this, or you receive an error, please refer to the troubleshooting section of the wiki.

Basic tutorial

Building on the model made in the previous section, try the following:

1. In the AnyLogic editor, drag in a new button from the Controls palette. Set its label to "set x", and in its Action field, type the following code:

pyCommunicator.run("x = 3.14");

The run function is used to send statements to Python that expect no response (e.g., variable assignments or import statements).

The passed string is what is sent to Python; it defines a new Python variable, x, which is set to the floating point number 3.14

2. Drag in another button, set its label to "get x", and in its Action field, type the following code:

double xValue = pyCommunicator.runResults(double.class, "x");
traceln(xValue);

The runResults function is used when you want to retrieve a value from Python.

The double class is passed to convert the desired variable ("x") to the expected Java type.

3. Run the model! You should first click the "set x" button, then press the "get x" button; afterwards, the number specified should be printed to the console.

If you press the "get x" button first, an error will be thrown because you tried to get the value of a variable before you defined it!

There are demos and example models provided with the library. Please review them to get some inspiration of possible use cases and to better understand the workflow.

Important disclaimers

• Pypeline is not part of the main AnyLogic product, and The AnyLogic Company is not obligated to provide support for users or to provide future support/updates
  • It's encouraged to make use of the community features available on here or elsewhere online
• Using Pypeline is not a substitute for Java, which still remains as the only native scripting language of AnyLogic
  • You can (and should) build models in the AnyLogic GUI exactly as before, making full use of AnyLogic's extensive native capabilities
• Pypeline will also add some computational overhead to your model and therefore may not be the best option if computational efficiency is a priority in your models

Contributing

Please feel free to contribute to Pypeline! For this purpose, pull requests are used - if you're not familiar with how to do this, consult GitHub Help for more information.

Any questions, issues, bug reports, or feature requests should be made on the "Issues" tab.

anylogic-pypeline's Issues

Using toJson for a filtered subset of agent population

Hi.

I'm not sure if this is the appropriate place to post this. If it is not, please refer me to somewhere I can receive help with Pypeline functions.

Currently I can use toJson to convert an agent population into a JSON string, which may then be used as input to a Python script.

The problem faced currently is that I would like to use only a subset of the agent population as input. I therefore filter the population, which gives a list of agents as output. When attempting to use toJson with this list, I get the following error:

"This agent is already defined as agent living in space 'Continuous' and can't have behaviour for space 'Discrete 2D'"

My code is as follows:

List routing_sub_population = filter(customers, c -> c.inState(Customer.statechart_state.awaiting_regular_delivery));
traceln("List: " + routing_sub_population);
String someListToJson =  pyCommunicator.toJson(routing_sub_population, true);
traceln("List: " + someListToJson);

Run Python scripts before each experiment run

I would like to run an external Python script before each AnyLogic experiment run. I tried to set a PyCommunicator object in the Simulation Experiment page through a variable as explained here ([https://www.anylogic.com/upload/pdf/q-and-a-with-writeups.pdf]).
I tried with the different constructors available, but when I run the model the variable is at null value.
In order to understand if I made some mistake in the definition the constructors of the PyCommunicator object, in the Main agent I defined a variable of type PyCommunicator copying the parameters of another working PyCommunicator object set by dragging and dropping from palette. Here the code:

//py is the PyCommunicator object created by drag and drop from Palette //py2 is the manually created PyCommunicator object PyCommunicator py2 = new PyCommunicator(py.enable,py.loadLastWorkingConfig,py.pythonCommandType,py.pythonCommand,py.pythonExecPath,py.throwErrorOnFailedAttempt,py.redirectPyOutput);

If I print at console the values of the parameters of the two objects I have this result:
Py object: Python 3.11.0 C:\...\python.exe Enable: true Load last config: false Python command type: PYTHON_PATH Python command: null Python executable path: C:\...\python.exe Throw error: false Redirect output: true Py2 object: null

What I am doing wrong?

Thanks

Use Pypeline in Anylogic version 8.5

In Q&A pdf file, it says "Regarding versions, the latest version of Pypeline (v1.4.0) is built for AnyLogic version 8.6. There’s nothing specific to the logic about v8.6 though, so with minimal modifications, you can adapt Pypeline to any version of AnyLogic." , but hwo to make these "minimal modifications" to use powerful Pypeline in version 8.5?
thanks a lot.

Anaconda or pip environment support

The current pypeline libary version allows me to select a python interpreter using a custom path.
However, when using virtual environments it is required to activate the environment first.

Hence, I am wondering wether a workaround exists to activate python environments such as conda or pip before executing the modell in AnyLogic?
Thank you in advance !

Issue with AnyLogic 8.8.2

I'm getting the following error under AnyLogic 8.8.2:

java.lang.ClassCastException: class [Com.anylogic.engine.presentation.ModelElement Descriptor Utils;
cannot be cast to class [Lcom.anylogic.engine.presentation.ModelElementDescriptor; ([Lcom.anylogic.engine.presentation.ModelElementDescriptorUtils;
and [Lcom.anylogic.engine.presentation.ModelElementDescriptor; are in unnamed module of loader 'app')

With AnyLogic 8.8.1 everything is working nicely. I'm suggesting not to (auto)update AnyLogic when using Pypeline.

By the way: thanks for this great library!

Missing user guide

Cannot locate full user guide. Can you please provide a path. Thanks

Pypeline and AnyLogic Cloud

Hi there! This is not really a problem, but rather a question. On the anylogic website it says that pypeline does not work with anylogic cloud. Are there any plans to integrate the library with the cloud? For example, I would personally find very useful to be able to export a model that uses python scripts to the cloud and then use the functionalities of the cloud API to run the model programmatically.

Reinforcment Learning

Hello, can you please tell me if you are planning to release a version with support for reinforcement learning? Thanks

Error in AnyLogic 8.8.5

I got this error, when I'm using AnyLogic 8.8.5 while I add this library, it will show me use AnyLogic 8.8.3 or higher.

Encoding error when trying to save a file on Windows

Hello!

I am getting an error when I try to execute the following code to save an agent's information to a json file:

String json = pyCommunicator.toJson(this);
pyCommunicator.run(
	"with open('./model-output/output-exp%s-rep%s.json', 'w', encoding='ISO-8859-1') as f:", 
	String.format("  f.write('''%s''')", json),
	"  f.close()"
);

Anylogic console:

at com.anylogic.engine.Engine.error(Unknown Source)
	at com.anylogic.engine.Agent.error(Unknown Source)
	at com.anylogic.engine.Utilities.error(Unknown Source)
	at com.anylogic.libraries.pypeline.PyCommunicator.run(PyCommunicator.java:583)
	at com.anylogic.libraries.pypeline.PyCommunicator.run(PyCommunicator.java:598)
	at prueba.Main.saveOutput(Main.java:640)
	at prueba.ParametersVariation.onEngineFinished(ParametersVariation.java:388)

It only occurs on Windows. I can run it on Linux with no errors, but when inspecting the output json file, I see that latin characters (like ñ or á) contained in the Anylogic model are replaced with question marks.
I think it is somewhat related to encodings, but I have already tried with utf-8, utf-16 and ISO-8859-1 on the open() code line this way:
"with open('./model-output/output-exp%s-rep%s.json', 'w', encoding='ISO-8859-1') as f:",

Thank you😄

Bug with Mac

Hello,

I have this error when I try to run Anylogic with pyCom :

Error during model startup:
root.pyCom:
Problem with setting up Python Subprocess
Caused by: Python process failed to properly start. This process needs to be started by a 'live' agent. Error message from process: 'Traceback (most recent call last):
File "/var/folders/ln/hpcn96rx41j38zx70j7m_r0h0000gn/T/pypeline-server_9841271844594326379.py", line 5, in
from json.decoder import JSONDecodeError
ImportError: cannot import name JSONDecodeError

Can you help me ?

Thank you in advance,

Mickael

No able to open the model examples

Hi , I was trying to open the models that are in .alp format in Anylogic. The case is that when I try to open it in Anylogic a message saying : invalid projec file format , is displayed.

Am I downloading the models incorrectly?

Thanks!

Agent population to json issue

Hi,

I'm back with some more questions! Thank you for the response on my previous question. I saw that it was my own mistakes rather than a bug. I really appreciate the feedback.

At the moment, I'm struggling with the details of translating an agent into a usable format in Python. Specifically, I am having issues with the parsing of certain variables. One of my agent's variable, for instance, is a Boolean variable. Python cannot interpret the false and true values contained in the Boolean variable.

I can successfully translate the agent into a json string using the toJson function.
A snippet of the string is given by (the details are unimportant, I would just like to show what the variable data looks like):

{"y_home" : 55.70862345859101, "_index" : 9, "value_of_time" : 4.7033915765776175, "trip_accepted" : false, "customer_to_serve" : null, "x_home" : 40.08944843268986, "incumbent_best" : null, "$SWITCH_TABLE$vrp_first_attempt$Occasional_Driver$statechart_state" : [ 1, 2, 3, 4 ], "_group" : 5}

Thereafter, the json string is used as the input to a function written in a Python script.

However, the following issue comes up:

Exception('The following line failed to parse (as strictly Python or JSON-parsed code):

The end of the message reads as follows:

Python reason: name \'false\' is not defined\n2. JSON reason: Expecting value: line 1 column 1 (char 0)')

I want to ask if there is a way to avoid sending ALL the agent's information when performing the toJson functionality. I know in a previous version all non-essential aspects of an agent was ignored (e.g. _text or _groups). If it were possible to ignore or leave out certain variable, it would be amazing.

Alternatively, do you have another way of dealing with the boolean variable, since false or true cannot be interpreted in Python. A similar issue persists with variable with a value of null.

I hope my description explains the issue sufficiently.

Hope to hear from you soon.

Kind regards,
Christian Malan

jsonifier library error

2 Errors after installing pyCommunicator in Anylogic 8.8.3

Python files missing

Hey Tyler,

Thanks a lot for reacting on my youtube comment. The new updated examples have really helped me get going.

An issue however with the updated pypeline 1.7.0 is that some of the examples are missing the needed python files to observe how they work. Namely the Demo - Agents and Populations with JSON.

Issue with reflectiveObjects

Type sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl is not accessible.

Error directly visibile in Java code in some of the examples.
Using JRE 11, toJson return error related to ParameterizedTypeImpl.
Using JRE 10, a method for handling String is missing.

fromJson for an int[][]

Hi,

Thank you for the quick feedback last time, it really helped a lot.

I am once again stuck.

I'm looking to convert the output from my python script run_vehicle_routing_sweep into an array of type int[][] in AnyLogic. In this case, the output from the python script is:

[
[-1, 0, 6, 4, 1, -1],
[-1, 2, 3, 5, -1]
]

I try to convert it into something useable in AnyLogic with the following code:

String sweep_result = pyCommunicator.runResults(String.class, String.format("run_vehicle_routing_sweep(%s,%s,%s)", customerJSON, vehicle_capacityJSON, depotJSON));

traceln("Sweep results: " + sweep_result);

int[][]route_order = pyCommunicator.fromJson(sweep_result, int[][].class);

traceln("Local route order: " + route_order[0]); 

This results in the output:

Sweep results: [[-1, 0, 6, 4, 1, -1], [-1, 2, 3, 5, -1]]
Local route order: [D@63d3832e

Do you have any suggestions?
I also tried to use a dictionary approach, but that gives additional errors (relating to java expecting double quotes, rather than the single quotes from the python dictionary).

I don't know if my approach is off, or if it is a bug. Either way, I would really appreciate some help.

Error

Hello.

While following your tutorial on installing Pypeline, I got the following error:

java.io.FileNotFoundException: C:\Program Files\AnyLogic 8.6 Professional\pypeline.properties (Access is denied)

Do you know how to get access? In addition, how where can I get the pypeline.properties file?

Thanks in advance for your help.

Regards,
Roberto

Deprecation warning from collections

Hi!

When running pypeline with python 3.7.7, I received the following warning:

DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working

from collections import Mapping

I think the solution is simple, just change at Pypeline Library.alp:
"from collections import Mapping",

To:

"try:",
    "from collections.abc import Mapping",
"except ImportError:",
    "from collections import Mapping",

This message appears because, as of python 3.3, some classes like Mapping were moved to the collections.abc module.

https://stackoverflow.com/questions/53978542/how-to-use-collections-abc-from-both-python-3-8-and-python-2-7

"Run and Runresults are not defined" problem

i am getting these errors upon execution, any idea ?

Description: The method runResults(Class, String) is undefined for the type Main._pyCommunicator_Population. Location: Trial Model/Main/level/button1 - Button

Description: The method run(String) is undefined for the type Main._pyCommunicator_Population. Location: Trial Model/Main - Agent Type

Error during model creation in AnyLogic

Hallo,

When I followed the Quick start guide creating model in AnyLogic,
there was something wrong when I run the model.

Error during model creation:
com/anylogic/engine/presentation/ColorConstants
Caused by: com.anylogic.engine.presentation.ColorConstants
java.lang.NoClassDefFoundError: com/anylogic/engine/presentation/ColorConstants
at com.anylogic.libraries.pypeline.Jsonifier.(Jsonifier.java:1955)
at model1.Main.instantiate_pyCommunicator_xjal(Main.java:125)
at model1.Main.instantiateBaseStructureThis_xjal(Main.java:263)
at model1.Main.(Main.java:246)
at model1.Simulation.createRoot(Simulation.java:155)
at model1.Simulation.createRoot(Simulation.java:1)
Caused by: java.lang.ClassNotFoundException: com.anylogic.engine.presentation.ColorConstants
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 6 more

Thanks for your support.
Best Regards

Syntax Error

when run through the command line, the code runs without error. However, when I use open the file with the pyCommunicator.run command an error comes out syntax error def (config: config). I thought that the problem is in the python version. However, after installing version 3.7, the error remained the same.

Reinforcement learning

Can you please tell me if there will be a version for training with reinforcement learning for python? And when will it be if it will be?

example model can not be opened

          im also having this issue. How was it solved?

Originally posted by @yuriypodvalny in #40 (comment)

Anylogic version

Hi!

Great work!
I have a full license Anylogic University 8.4.0. While loading the library I got the following message "This model was created with newer version of Anylogic. ALP file version: (8.5.1) Upgrade Anylogic to the latest version to open this model."

I changed already the version number within the code of the .alp file "Pypeline Library.alp". With this modification, I can open the file in 8.4. But the same error occurs when loading the .jar file.
Do you know how to insert the library for Anylogic University version 8.4.0?

Update: Same message when inserting the library in Anylogic 8.5.1 PLE.

Thanks a lot!
Jop

Run AnyLogic from PyPeline

Hello,

Is there a way to run AnyLogic simulation with a command from the PyCommunicator?
If yes - Is there a way to provide an arguments to the simulation from PyCommunicator?

Is there an option to observe the simulation results when ended from the PyCommunicator?

Thanks and Regards,
Elad

how can you start the simulator and initialize the data using a python file?

I need to receive data from the simulation and pass it to python and this should all happen recursively. Is this possible with this library? Thanks.

Pycommunicator not dragging onto the experiment page in anylogic

I am trying to use py-communicator with any-logic, but I am unable to access pycommunicator library in any of my experiment windows like compare runs or parameter variation experiment. I basically want to run a python code after several runs of a compare runs experiment but py communicatior cannot be dragged.It is working well in the main interface. I am using the anylogic university 8.5.2 version. Basically if can drag pycommunicator to the experiment window, I can create a button in the experiment window which when clicked will run the python code and print the results. But pycommunicator is not getting recognized in the experiment window. So I was trying to call the function with pycommunicator in main to the experiment window but I am not understanding how to call it either. I need this to work please clarify

Unable to load a pickled object

We are using a graph to do some pathfinding for us in python. The graph is fairly large, so we are generating the graph elsewhere and pickling the python object to be loaded later. AnyLogic is one of our test beds for testing the output of the pathfinding algorithms. We need to load the pickled object but have been unsuccessful. The error given:

Failed to run python code; feedback: AttributeError("Can't get attribute 'CustomClass' on <module '__main__' from 'C:\\\\Users\\\\REDACTED\\\\AppData\\\\Local\\\\Temp\\\\pypeline-server_13616648004425206534.py'>")
	at com.anylogic.engine.Engine.error(Unknown Source)

I've imported the class in the error message using a from statement. I'm running the pickle.load command right in AnyLogic.

pyComm.run("with open('python_files/package/src/package/pickled_object.pkl', 'rb') as file:",
" custom_class_object = pickle.load(file)");

I'm able to declare a new object of the class as well.

pyComm.run("custom_class_object=CustomClass()");

Is there a limitation with "where" on my computer pypeline actually runs? This is my first guess based on the output of the error showing it's running main from a temp directory.

Any help is appreciated!

Error with setting up python subprocess (Macbook)

Hi,

I have the following error when trying to play the "Passing data types 2 demo". I did dowload the pypeline library. Do you have an idea how to solve this?

root.pyCommunicator:
Problem with setting up Python Subprocess:
Caused by: Input returned null; Error stream line reads: Traceback (most recent call last):
File "/var/folders/xt/f4729vqn5zn79c1942d4_tsw0000gn/T/pypeline-server_6617826609874667048.py", line 5, in
from json.decoder import JSONDecodeError
ImportError: cannot import name JSONDecodeError

Issues with Python on Mac

In the "Python Executable Path" part of the Pypeline module, I added where my Python exe file is on my mac ("usr\bin\python3") and I'm getting this error when I try to run it:
"Error during model startup:

root.pyCommunicator:

Problem with setting up Python Subprocess

Caused by: Python process failed to properly start. This process needs to be started by a 'live' agent. Error message from process: 'xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun"

I tried using just "usr\bin" as well, but that didn't work either.
It did run when I used this path "/Users/Erin/Applications/Python 3.9/bin" however that points to an python Alias file and not the exe. And it's still not working because it can't find the python packages needed for my project (even though they are installed on my computer)

Python and Anylogic Connection

Hello dear t-wolfeadam,

I am trying to connect Anylogic and Python. I know there is a Pipeline tool in Anylogic but I intend to get the non-stop generated output to Anylogic block in real time. I have searched for it and I encountered Python's socket library and Anylogic's REST API. I am confused about which one to use. By the way, I want to get the data from an IoT device as an input to Python.