krumiaa / mindmaker Goto Github PK

I want to do multi-agent reinforcement learning, But the Mindmaker Remote ML Server can't be downloaded, and there's no repository on GitHub. Please can you re-upload, I need to use this software urgently.

Hi!

I setup the MARL as described in the github. However, after a while (10 seconds) the second agent training stops working with the print "Disconnected from Unreal Engine. Exiting Mindmaker". The first agent trains fine.

Do you have any idea why this might happen?

I did the following steps:

• Second agent mindmaker blueprint with the new address & port "http://localhost:3001"
  
  
• Second agent SocketIO address & port changed to "http://localhost:3001"
  
• In the python source code, changed the "3000" to "3001".
  
• Launched the second duplicate python source code (with 3001), then Unreal. The mindmaker.exe window also appears. Both agents start to train.
• After around 10-15 seconds the second agent python training stops due to a disconnection from Unreal Engine.

My program ran perfectly fine and without editing any blueprint that was using the SocketIO client suddenly my game started crashing within a few seconds of launching it. The MindMaker executable does open and prints out until

but even before those two lines are printed out the engine already crashes.

If I don't place the blueprint using the SocketIO client in the scene, everything runs fine, but as soon as code from that blueprint is either compiled OR executed the engine crashes. I can't even remove the SocketIO client from the blueprint without the engine crashing immediately.

Any idea what might be causing this and how I could fix it?

This is what Unreal Engine shows me as a crash report.:

Fatal error!

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION writing address 0x00007ffba15cea48

0x00007ffba6e076e0 UE4Editor-SocketIOLib.dll!sio::client_impl::close() [D:\build++Portal\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:169]
0x00007ffba6df4617 UE4Editor-SocketIOLib.dll!sio::client_impl::~client_impl() [D:\build++Portal\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:87]
0x00007ffba6df458d UE4Editor-SocketIOLib.dll!sio::client::~client() [D:\build++Portal\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\sio_client.cpp:42]
0x00007ffba15b1a98 UE4Editor-SocketIOClient.dll!SharedPointerInternals::TReferenceControllerWithDeleter<sio::client,SharedPointerInternals::DefaultDeletersio::client >::DestroyObject() [D:\RocketSync\4.27.0-17155196+++UE4+Release-4.27\Working\Engine\Source\Runtime\Core\Public\Templates\SharedPointerInternals.h:116]
0x00007ffba15a8177 UE4Editor-SocketIOClient.dll!FSocketIONative::~FSocketIONative() []
0x00007ffba15b1a67 UE4Editor-SocketIOClient.dll!SharedPointerInternals::TReferenceControllerWithDeleter<FSocketIONative,SharedPointerInternals::DefaultDeleter >::DestroyObject() [D:\RocketSync\4.27.0-17155196+++UE4+Release-4.27\Working\Engine\Source\Runtime\Core\Public\Templates\SharedPointerInternals.h:116]
0x00007ffba15acb0b UE4Editor-SocketIOClient.dll!UE4Function_Private::TFunction_CopyableOwnedObject<<lambda_97e8ba7979b1b36926b54dc4cade2d91>,0>::scalar deleting destructor'() [] 0x00007ffba6f19875 UE4Editor-CoreUtility.dll!TAsyncQueuedWork<void>::scalar deleting destructor'() []
0x00007ffbe9de5298 UE4Editor-Core.dll!UnknownFunction []
0x00007ffbea26ff0b UE4Editor-Core.dll!UnknownFunction []
0x00007ffbea267050 UE4Editor-Core.dll!UnknownFunction []
0x00007ffca5fb7034 KERNEL32.DLL!UnknownFunction []
0x00007ffca6dc2651 ntdll.dll!UnknownFunction []

Crash in runnable thread PoolThread 9
Crash in runnable thread PoolThread 4
Crash in runnable thread PoolThread 0

After clicking create project for UE5 (previously it would allow project creation for other versions, and the free plugin was also available for other versions, but now it shows support for only UE5 on the marketplace), it shows the following attached error.

However, when I download and run the example project (cartpole), it runs as it should but on UE4.25, as Ai plugin is manually present in the plugins folder.
I have tried one too many things on different systems, but now I am more confused than ever if the error is environment on my system related, related to the changes in the 'supported versions' of the Mindmaker, or something else.

Hey, it's me again.

I've gotten a whole lot further since the last time I posted :)
This time I wanted to ask if passing in parameters with different ranges is allowed or detrimental to the Learning Engine working?
Apart from all of my unsigned floating point variables I also need to pass in a few unit vectors which I've split into float values.
The issue with this is, that obviously these can contain values down to -1.0

Is this problematic or can I just use them like that as long as I define the right range?

i have bought the DRL engine and im messing around with the match to sample example. the only problem is that i dont understand how the agent visualization works, to be more specific, the display agent actions function. ill be completely honest that i know only the basics of neural networks. if anyone can give me even a brief explanation on how the visualization works ill be grateful.

please could you share your plans for supporting unreal engine 5 ?

Hello,

in your cartpole example, whenever the Cylinder falls below a certain pitch, the cylinder and cart are reset to their original position and the training commences. How did you implement this, because I couldn't find the code for it in the example.

In my case, if my car drives into a wall, I'd like to give it a negative reward to punish it for that behavior and reset its position back to the start of the race track. When that happens, do I pause sending observations until the car is back at its location and ready to go or do I have to reset the training episode counter in some way?

It would be extremely helpful if you could shed some light on this for me :)

best regards,

FLOROID

I'm setting the done flag when I reach an early termination event with the expectation that the episode will end. I track the number of steps and episodes within UE but it seems even after setting the done flag to true and passing it back through mindmaker, the episode continues until the defined number of timesteps is complete - how can I terminate early? I'm using PPO2.

I am currently facing difficulty in using a saved model file (pth file) in the Unreal Engine 5 MindMaker AI framework. Specifically,

I am unable to build a Windows standalone game using the saved model file. Can someone please guide me on how to effectively use a saved model file in the MindMaker framework to build a Windows standalone game?

Any help or pointers would be greatly appreciated. Thank you!

Hello there,

I'm currently working on a project for my bachelors paper and
want to train an AI agent using MindMaker.
When looking at the example project I noticed that in the "Make Observations" function
two Integer values are turned into strings appended in a [Integer, Integer] structure and passed into MindMaker.
Could I also pass float values to MindMaker here or are these values supposed to be either 0 or 1?

Thanks in advance.

Hello, I'm trying to test the plugin on the examples map but the training doesn't start. I'm using UE5

when I press the play button nothing happens. This is what I read in the output log:

LogBlueprintUserMessages: [cart_2] MindMaker Launched
LogBlueprintUserMessages: [cart_2] Waiting for Connection from Learning Engine
PIE: Server logged in
PIE: Play in editor total start time 1.347 seconds.
SocketIO: SocketIO Invalid appears to have lost connection, reconnecting attempt 0 with delay 5000
SocketIO: SocketIO Invalid appears to have lost connection, reconnecting attempt 1 with delay 7500
SocketIO: SocketIO Connected with session: 353587e91bee425d8acada58836da487
SocketIO: SocketIO 353587e91bee425d8acada58836da487 connected to namespace: /
SocketIO: Warning: Attempted to connect to http://localhost:3000 when we're already connected. Request ignored.
LogOutputDevice: Warning: Script Stack (3 frames):
cart_BP_C.LaunchMindMaker
cart_BP_C.ExecuteUbergraph_cart_BP
cart_BP_C.BndEvt__SocketIOClient_K2Node_ComponentBoundEvent_0_SIOCOpenEventSignature__DelegateSignature
LogOutputDevice: Error: Ensure condition failed: Lhs.Array.Num() == Lhs.InitialNum [File:C:\Users\epilo\Documents\GitHub\MindMaker_UE5\Engine\Source\Runtime\Core\Public\Containers\SparseArray.h] [Line: 972]
Container has changed during ranged-for iteration!
LogStats: FDebug::EnsureFailed - 0.000 s
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogWorld: BeginTearingDown for /Game/MindMaker/Examples/Assets/Maps/UEDPIE_0_Cartpole
LogWorld: UWorld::CleanupWorld for Cartpole, bSessionEnded=true, bCleanupResources=true
SocketIO: SocketIO Disconnected 353587e91bee425d8acada58836da487 reason: CLOSE_REASON_NORMAL

Debugging the code it seems the function emit gets called but, after that I'm kind of lost.

if I look at netstat in cmd the process seems connected to localhost port 3000 correctly.

Maybe I'm missing something simple. Are the examples supposed to work out of the box or do I have to change any parameter?

I run the code of mindmaker server in the pycharm, but it can't build connection between mindmaker and ue4. UE4 suggests the following error " SocketIO: SocketIO Invalid appears to have lost connection, reconnecting attempt 0 with delay, 5000". I want to know when using mindmaker server, how to set up the blueprint nodes in ue4.

I've set everything up in my project now and the agent is training without any errors, however when the Evaluation phase starts, he will continue executing the same action he executed in the first evaluation episode for every evaluation episode until the evaluation is done.
What could be causing this and how could I go about fixing this issue? Until I can actually properly evaluate what the agent is learning I can't really make use of the training, so this would be very crucial to fix for me.

Hello,

I've been enjoying the plugin a lot, however I was trying to figure out if it's possible to have a non-MARL multi-agent setup, specifically with PPO.

Buy this I mean having multiple agents collect experience for one neural network, which is a very common technique for networks like PPO. I found a section in the wiki talking about setting up multiple clients in order to train multiple networks, but I am trying to use multiple agents to train one network.

Is this possible with the plugin, and how if so?

I need to be able to run 2+ separate instances of MindMaker on a single project at the same time. How do I accomplish this?

Hello there ^^

I am currently trying to set up my AI Agent for Autonomous Driving.
I've set up my Make Observations Functio and the Check Reward function and I am now wanting to bind the actions chosen by the Algorithm to their respective actions in unreal engine. For example having the action with the iterator 1 increase the throttle of the car.
Now I'm trying to figure out when to do that. Do I execute those actions inside the Receive Action function before Observations are made using the "Action Selected" Integer Value? Also, does the range of values for Action Selected start at 0?

And as a final question - does the Mindmaker executable automatically call the Receive Action function or do I have to call it anywhere because I couldn't find any place in the example project where the function was being called?

Best regards,

Flo

I am trying to build a basic RL project with mindmaker plugin. The project contains an agent and a target, agent goal is to reach the target by taking (up, down, left, right) action and the agent will get a reward (+10) when it reaches the target otherwise -1 reward on each action. Ball is an agent and cube is the target in the project.

I am sharing the same project, if someone wants to download it to check the error. The structure of blueprint is very much similar to the inbuilt cartpole project. Kindly have a look.

I am using UE5.1.

https://drive.google.com/file/d/1SSVsYukVMrxQ076r3BdY0aI6KCM92pfI/view?usp=share_link

While launching the mindmaker this error is coming.

MindMaker running, waiting for Unreal Engine to connect
(6564) wsgi starting up on http://0.0.0.0:3000
(6564) accepted ('127.0.0.1', 51323)
Connected To Unreal Engine
False
4
discrete action space
C:\Users\aiina\Documents\Unreal Projects\Rl_test\Content\MindMaker\dist\mindmaker\gym\spaces\box.py:74: UserWarning: WARN: Box bound precision lowered by casting to float32
save model value: true
load model value: false
Total Training Episodes: 1000
alg selected: PPO
use custom: true
Exception in thread Thread-5:
Traceback (most recent call last):
  File "threading.py", line 917, in _bootstrap_inner
  File "threading.py", line 865, in run
  File "site-packages\socketio\server.py", line 720, in _handle_event_internal
  File "site-packages\socketio\server.py", line 745, in _trigger_event
  File "MindMakerDRLStableBv3.py", line 606, in recieve
  File "ast.py", line 46, in literal_eval
  File "ast.py", line 35, in parse
  File "<unknown>", line 0

    ^
SyntaxError: unexpected EOF while parsing

What is going wrong in the project?
How to solve it?

Hi, I bought and modified the MindMaker Single Node Project. I am trying to output the training statistics to tensorboard. However, regardless of my input to the tensorboard_log parameter, no files are generated. Would you have any ideas as to why?

Thank you.

I am creating platformer game with RL agent. It is mario-like game, where AI player gets array of nearby tiles. This is 1D array with 90 elements (in my game i am transforming it into 2D (9,10) Array ).

I started really simple, but when got to moment when i starts MindMaker i got issue:
This is log from mindmaker console

(censored path)...Project\Content\MindMaker\dist\mindmaker\tensorboard\compat\tensorflow_stub\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
MindMaker running, waiting for Unreal Engine to connect
(14476) wsgi starting up on http://0.0.0.0:3000
(14476) accepted ('127.0.0.1', 2927)
Connected To Unreal Engine
False
6
discrete action space
(censored path)...Project\Content\MindMaker\dist\mindmaker\gym\logger.py:30: UserWarning: ←[33mWARN: Box bound precision lowered by casting to float32←[0m
save model value: false
load model value: false
alg selected: PPO2
use custom: true
Exception in thread Thread-4:
Traceback (most recent call last):
  File "threading.py", line 926, in _bootstrap_inner
  File "threading.py", line 870, in run
  File "site-packages\socketio\server.py", line 679, in _handle_event_internal
  File "site-packages\socketio\server.py", line 708, in _trigger_event
  File "mindmaker.py", line 716, in recieve
UnboundLocalError: local variable 'policy_kwargsval' referenced before assignment

I also catch .json that is being sent to mindmaker server:

{
    "observations": "[0,0]",
    "trainepisodes": 1000,
    "evalepisodes": 100,
    "maxactions": 6,
    "loadmodel": "false",
    "savemodel": "false",
    "algselected": "PPO2",
    "customparams": "true",
    "modelname": "platformer_game_ai_model",
    "conactionspace": false,
    "done": false,
    "actionspace": "6",
    "observationspace": "low=np.zeros(90), high=np.full(90,10), dtype=np.float32",
    "disactionspace": true,
    "a2cparams": {
        "gamma": 0.9900000095367432,
        "n_steps": 5,
        "vf_coef": 0.25,
        "ent_coef": 0.009999999776482582,
        "max_grad_norm": 0.05000000074505806,
        "learning_rate": 0.000699999975040555,
        "alpha": 0.9900000095367432,
        "epsilon": 9.999999747378752e-06,
        "lr_schedule": "constant",
        "verbose": 0,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "None",
        "network_arch": "[8, 'lstm', 64, 32]",
        "act_func": 0,
        "policy": 0
    },
    "acerparams": {
        "gamma": 0.9900000095367432,
        "n_steps": 5,
        "ent_coef": 0.009999999776482582,
        "max_grad_norm": 10,
        "learning_rate": 0.000699999975040555,
        "alpha": 0.9900000095367432,
        "lr_schedule": "linear",
        "verbose": 1,
        "num_procs": "None",
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "1",
        "network_arch": "[64, 64, 32]",
        "rprop_alpha": 0.9900000095367432,
        "q_coef": 0.5,
        "rprop_epsilon": 4.999999873689376e-05,
        "buffer_size": 5000,
        "replay_ratio": 4,
        "replay_start": 1000,
        "correction_term": 10,
        "trust_region": "True",
        "delta": 1,
        "act_func": 0,
        "policy": 0
    },
    "acktrparams": {
        "gamma": 0.9900000095367432,
        "n_steps": 5,
        "ent_coef": 0.009999999776482582,
        "max_grad_norm": 0.5,
        "learning_rate": 0.25,
        "lr_schedule": "linear",
        "verbose": 1,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "1",
        "network_arch": "[64, 64, 32]",
        "nprocs": "None",
        "vf_coef": 0.25,
        "vf_fisher_coef": 1,
        "kfac_clip": 0.0010000000474974513,
        "async_eigen_decomp": "False",
        "kfac_update": 0,
        "gae_lambda": "None",
        "act_func": 0,
        "policy": 0
    },
    "dqnparams": {
        "learning_rate": 0.0005000000237487257,
        "verbose": 1,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "1",
        "layers": "[64, 64]",
        "buffer_size": "50000",
        "exploration_fraction": 0.10000000149011612,
        "exploration_final_eps": 0.019999999552965164,
        "exploration_initial_eps": 1,
        "train_freq": 1,
        "batch_size": 32,
        "double_q": "True",
        "learning_starts": 100,
        "target_network_update_freq": 500,
        "prioritized_replay": "False",
        "prioritized_replay_alpha": 0.6000000238418579,
        "prioritized_replay_beta0": 0.4000000059604645,
        "prioritized_replay_beta_iters": "None",
        "prioritized_replay_eps": 6.000000212225132e-06,
        "param_noise": "False",
        "act_func": 0,
        "policy": 0,
        "gamma": 0.9900000095367432
    },
    "ppo2params": {
        "gamma": 0.9900000095367432,
        "verbose": 1,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "None",
        "layers": "[64, 64]",
        "lam": 0.949999988079071,
        "n_steps": 128,
        "ent_coef": 0.009999999776482582,
        "learning_rate": 0.0002500000118743628,
        "vf_coef": 0.5,
        "max_grad_norm": 0.5,
        "nminibatches": 4,
        "noptepochs": 4,
        "cliprange": 0.20000000298023224,
        "cliprange_vf": "None",
        "act_func": 0,
        "policy": 0
    },
    "sacparams": {
        "gamma": 0.9900000095367432,
        "learning_rate": 0.0003000000142492354,
        "verbose": 1,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "None",
        "layers": "[64, 64]",
        "buffer_size": "50000",
        "train_freq": 1,
        "batch_size": 64,
        "learning_starts": 100,
        "tau": 0.004999999888241291,
        "ent_coef": "auto",
        "target_update_interval": 1,
        "gradient_steps": 1,
        "target_entropy": "auto",
        "action_noise": "None",
        "random_exploration": 0,
        "act_func": 0,
        "policy": 0
    },
    "td3params": {
        "gamma": 0.9900000095367432,
        "learning_rate": 0.0003000000142492354,
        "verbose": 1,
        "tensorboard_log": "None",
        "_init_setup_model": "True",
        "full_tensorboard_log": "False",
        "seed": "None",
        "n_cpu_tf_sess": "None",
        "layers": "[64, 64]",
        "buffer_size": 50000,
        "train_freq": 100,
        "batch_size": 128,
        "learning_starts": 100,
        "tau": 0.004999999888241291,
        "gradient_steps": 100,
        "action_noise": "None",
        "random_exploration": 0,
        "policy_delay": 2,
        "target_policy_noise": 0.20000000298023224,
        "target_noise_clip": 0.5,
        "act_func": 0,
        "policy": 0
    },
    "reward": 0
}

tbh i need this to graduate my studies, so help would be welcome

python-socketio==5.x
python-engineio==4.x
in this version, ue4 can’t connent to server, can you list your pip list, It will help me a lot !

Now that ue 5 preview is released, its api should be fixed by now. Can you please update this plugin to ue5?

I tried using ue5 plugin you provided, but it can't find an entry called IsSelectedInEditor.

I also tried rebuilding the plugin using UnrealBuildTool.exe, but it failed on SocketIOLib. I think it is caused by ue5 preview has different impl from ue5 ea.

Hi, im trying to add DRL to a game i made, however i have found no resourcces anywhere that outline how DRL can be used on a custom game, any help will be greatly appreciated

I'm trying to integrate this project into UE5, Preview 1. It's all going well so far and thank you for releasing this! I am running into an issue, though, after calling the Launch Mind Maker node but before the receive actions or receive observations are called:

Traceback (most recent call last):
  File "threading.py", line 926, in _bootstrap_inner
  File "threading.py", line 870, in run
  File "site-packages\socketio\server.py", line 679, in _handle_event_internal
  File "site-packages\socketio\server.py", line 708, in _trigger_event
  File "mindmaker.py", line 271, in recieve
KeyError: 'done'

It seems to have something to do with how I package my parameter structure. I've tried to keep this as close to the cartpole example as possible but I may have missed something vital. I tried to look for the mindmaker.py source code but as it seems to be unavailable I'm posting here.

Any help is much appreciated!

Hi,

I got an error exclusively when using PPO2 with full tensorboard logs = true. It happens after around 300 steps:

Here are my settings:

It trains successfully while full tensorboard logs = false.
The agent also trains successfully using other algorithms with full tensorboard logs = true. Not sure why PPO2 only have this problem.

Note that I still get tensorboard logs just not the full data, so it's not a major issue.

I'm having problems with the action noise syntax. I've looked at the documentation for stable baselines, as well as through some of the source python code, but to no avail. What is the allowed syntax that I can use within this parameter? Thank you.

I downloaded both plugins with UE 5.2 on my system however when I tried to create project from the Deep RL the editor could not load the base plugin necessary since it is only compatible with up to 5.1

Hiya,

I'm using UE version 5.0.3 and when I load up the cart example (with no changes, created from the UE marketplace), MindMaker.exe starts but has this ImportError straight away and closes.

Any ideas or suggestions?

Hey @krumiaa ,

I am currently working with MindMaker to build a game AI (using DQN). This AI should search and catch an opponent. To do this it utilizes basic FPS inputs (moving forward, backward, left and right, turning left, right, up and down and so on). The first challenge here is to get the Agent to properly use these controls to navigate the environment. After some trial and error this actually works. The problem now is, that most actions are not actually helpful in achieving the goal of catching the opponent.

Usually in machine learning an episode would be one game session. So for example in a game of Chess it would be one whole match. Now the problem is, that with the current UE4 implementation the agent only takes one action and then immediatly emits a reward. I would like to change that, but don't know how I would go about doing that.
Currently rewards and observations are emitted at the same time, signaling the end of an episode. I could technically only emit these two every 20 actions, but then the network would not reviece the new game state after every action. So what I would like to do is take let's say 20 actions and emit the new observations after each action. And at the end of the 20 actions the cummulative rewards of all 20 actions are emitted.

In theory this should enable the agent to act with intend and planning over multiple actions. I hope this makes sense to you. If you could point me in the right directions this would be great.

Cheers

Abort signal received

ucrtbase
ucrtbase
ucrtbase
VCRUNTIME140_1
VCRUNTIME140_1
VCRUNTIME140_1
ntdll
ntdll
ntdll
KERNELBASE
VCRUNTIME140
UnrealEditor_SocketIOLib!std::invoke<<lambda_ca95016d73eb338498511f9be7c424bb> &,bool,std::shared_ptr<std::basic_string<char,std::char_traits,std::allocator > const > const &>() [d:\build\AutoSDK\Sync\HostWin64\Win64\VS2019\14.29.30147\INCLUDE\type_traits:1534]
UnrealEditor_SocketIOLib!sio::packet_manager::encode() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_packet.cpp:485]
UnrealEditor_SocketIOLib!sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::on_ping() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:607]
UnrealEditor_SocketIOLib!sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::on_decode() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:639]
UnrealEditor_SocketIOLib!sio::packet_manager::put_payload() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_packet.cpp:529]
UnrealEditor_SocketIOLib!sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::on_message() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:560]
UnrealEditor_SocketIOLib!std::_Func_impl_no_alloc<std::_Binder<std::_Unforced,void (__cdecl sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::*)(std::weak_ptr,std::shared_ptr<websocketpp::message_buffer::message<websocketpp::mes() [d:\build\AutoSDK\Sync\HostWin64\Win64\VS2019\14.29.30147\INCLUDE\functional:823]
UnrealEditor_SocketIOLib!std::_Func_class<void,std::weak_ptr,std::shared_ptr<websocketpp::message_buffer::messagewebsocketpp::message_buffer::alloc::con_msg_manager > >::operator()() [d:\build\AutoSDK\Sync\HostWin64\Win64\VS2019\14.29.30147\INCLUDE\functional:869]
UnrealEditor_SocketIOLib!websocketpp::connectionwebsocketpp::config::asio_client::handle_read_frame() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\websocketpp\websocketpp\impl\connection_impl.hpp:1099]
UnrealEditor_SocketIOLib!websocketpp::transport::asio::connectionwebsocketpp::config::asio_client::transport_config::handle_async_read() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\websocketpp\websocketpp\transport\asio\connection.hpp:897]
UnrealEditor_SocketIOLib!asio::asio_handler_invoke<asio::detail::binder2<websocketpp::transport::asio::custom_alloc_handler<std::_Binder<std::_Unforced,void (__cdecl websocketpp::transport::asio::connectionwebsocketpp::config::asio_client::transport_config() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\handler_invoke_hook.hpp:69]
UnrealEditor_SocketIOLib!asio::detail::wrapped_handler<asio::io_context::strand,websocketpp::transport::asio::custom_alloc_handler<std::_Binder<std::_Unforced,void (_cdecl websocketpp::transport::asio::connection<websocketpp::config::asio_client::transport() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\detail\wrapped_handler.hpp:98]
UnrealEditor_SocketIOLib!asio::detail::read_op<asio::basic_stream_socketasio::ip::tcp,asio::mutable_buffers_1,asio::mutable_buffer const *,asio::detail::transfer_at_least_t,asio::detail::wrapped_handler<asio::io_context::strand,websocketpp::transport::asio() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\impl\read.hpp:284]
UnrealEditor_SocketIOLib!asio::detail::completion_handler<asio::detail::rewrapped_handler<asio::detail::binder2<asio::detail::read_op<asio::basic_stream_socketasio::ip::tcp,asio::mutable_buffers_1,asio::mutable_buffer const *,asio::detail::transfer_at_leas() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\detail\completion_handler.hpp:69]
UnrealEditor_SocketIOLib!asio::detail::asio_handler_invoke<asio::detail::binder2<asio::detail::read_op<asio::basic_stream_socketasio::ip::tcp,asio::mutable_buffers_1,asio::mutable_buffer const *,asio::detail::transfer_at_least_t,asio::detail::wrapped_handl() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\detail\wrapped_handler.hpp:231]
UnrealEditor_SocketIOLib!asio::detail::win_iocp_socket_recv_op<asio::mutable_buffers_1,asio::detail::read_op<asio::basic_stream_socketasio::ip::tcp,asio::mutable_buffers_1,asio::mutable_buffer const ,asio::detail::transfer_at_least_t,asio::detail::wrapped() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\detail\win_iocp_socket_recv_op.hpp:98]
UnrealEditor_SocketIOLib!asio::detail::win_iocp_io_context::do_one() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\detail\impl\win_iocp_io_context.ipp:420]
UnrealEditor_SocketIOLib!asio::io_context::run() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\ThirdParty\asio\asio\include\asio\impl\io_context.ipp:61]
UnrealEditor_SocketIOLib!sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::run_loop() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\MindMaker\Source\SocketIOLib\Private\internal\sio_client_impl.cpp:261]
UnrealEditor_SocketIOLib!std::thread::_Invoke<std::tuple<std::_Binder<std::_Unforced,void (__cdecl sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client >::)(void),sio::client_impl<websocketpp::clientwebsocketpp::config::asio_client > *> () [d:\build\AutoSDK\Sync\HostWin64\Win64\VS2019\14.29.30147\INCLUDE\thread:56]
ucrtbase
kernel32
ntdll

Hi @krumiaa

2 questions:

Just recently purchased MindMaker: Deep Reinforcement Learning (SingleNode) from UE marketplace and had a look at your Cartpole example, it seems a connector is missing in cart_BP?

Also, from my understanding, the "Done" boolean is used to notify the RL engine that the current episode has ended, is that correct?

Thanks.

Trying to save and load the training model but getting this error. Where would I change the location it is trying to use? Already tried running MindMaker as an administrator.

Out of the box the UE4 example projects all work, however in ue5 the learning engine doesn't seem to launch, and when launched manually doesn't seem to connect. Upgrading the ue4 project to 5 blows up on compile for a number of obvious reasons. Copying the dist folder from a working copy into the ue5 project also doesn't seem to affect this issue.

I'm trying to run my training for 10k iterations (and that is set in "Trainepisodes" in the structure) but the training seems to stop short at ~391. Is there any reason it might quit early?

Hi,

I am testing MindMaker example project with UE5.
I would like to know that default setting(e.g. PPO2, number of training episode, ..) of this example project is enough to train the cart pole RL agent? I have tried training about over 30 times but never succeeded.