BPIC_2017 running with fuzzy resource profiles and extraneous discovery method switched to naive fails about simod

[david-chapela]

@iharsuvorau

This issue should be solved with the pull request I made on pix-framework (attached to this issue).

Check it out when you can, and accept it if you don't see anything weird. And run the benchmarks again, please.

[david-chapela]

Hi @iharsuvorau, we actually never tried this case because the tests of Fuzzy+Extraneous we run were without optimization (in extraneous delays), so Prosimos wasn't being called within this package.

I don't have much time these days, could you take a look at it? I checked the code and the simulation model (including the specific fuzzy parameters) are being passed correctly to the ExtraneousDelays package, so I can't easily see a reason why it would run the Crisp version of Prosimos.

Key points in the workflow of how it works:

• In here Simod creates the SimulationModel instance that ExtraneousDelays uses, and puts there the simulation parameters (using ._to_prosimos_format() function here).
• I've checked, and this function DOES add the model_type: "FUZZY" to these dict with the parameters, so it should be fine.
• Then ExtraneousDelays package enhances these parameters by adding the timer events modeling the delays in here. However, this just adds an extra field, so should be fine too.
• Then, it simulates with Prosimos with these parameters dumped in a file.

My idea would be to check the parameters file of one of these ExtraneousDelays iterations (debugging) and see if it has the model_type = "FUZZY" element just to be sure that the json file is correct. If Prosimos is running Crisp I would imagine that at some place we are missing this model_type = "FUZZY", but can't imagine where without debugging.

[david-chapela]

@iharsuvorau I just checked running it in local and checking the JSON files, and the parameters are Crisp, I don't really understand how it is happening, if you can take a look at it. The JSON files are:

• In all iterations of Control-flow optimization: Fuzzy.
• In all iterations of ResourceModel optimization: Fuzzy.
• In the first iteration of ExtraneousDelay: Crisp.

Something is happening in the call to ._to_prosimos_format() when preparing the ExtraneousDelayOptimizer.

[iharsuvorau]

Hi @orlenyslp. We're having an issue with Simod and its dependencies. So far, it seems the issue is the one we once discussed with you, when CRISP calendar discovery (that we moved to pix-framework repo) had been calculating negative duration, remember?

This is the place where it's happening, https://github.com/AutomatedProcessImprovement/pix-framework/blob/main/src/pix_framework/discovery/resource_calendar_and_performance/crisp/resource_calendar.py#L680C13-L680C13.

Here's the screenshot from the debugger with parameters, because this bug isn't so easy to reproduce:

You can see, the total_duration is negative. Is it correct behavior for the CRISP calendar discovery? You've been fixing this bug before, right?

[iharsuvorau]

@orlenyslp However, the bug we're facing is probably a different one, though, in the same function.

On the screenshot below, we can see that i = -3 when self.work_intervals[c_day] length is only 2 items. When the function tries to get self.work_intervals[c_day][-3] it's actually trying to get the third result from the end of the list. Is it an expected behavior?

[iharsuvorau]

Hi @david-chapela. Sorry, my bad. I'd been working on this dumping/parsing of resource models to and from a dictionary 3 weeks ago in pix-framework. I didn't realize that we use .from_dict internally in Simod also.

Sure, thank you! I'm re-running all the benchmarks now because this affected everything after the resource optimization step in Simod, including extraneous that we've been examining for the last week.

Still, do we need to deal with negative the duration in RCalendar? Is it okay to be negative? Should we mark it as a bug?

[david-chapela]

@iharsuvorau yeah, it took me a while to get to see that it was because of how we read the ResourceModel. I was first checking all the files, and all of them were good.

Don't worry about the benchmarks of previous weeks, the extraneous delays were not optimized in those cases (we were running the direct optimization), so there should be no difference in the result. I think it only affected this last run where we optimize the ExtraneousDelays with 20 iterations.

[david-chapela]

Regarding the negative values... I wouldn't go deep for now. It could be possible that the reason behind that negative case is due to running Prosimos with Crisp but with data that was actually belonging to Fuzzy.

Maybe the best decision would be to inspect in detail what's happening to check if it's due to a bug, or to running Prosimos-crisp using fuzzy parameters. However, that would probably take a lot of time for just a check (we're not really getting a bug running it under a normal scenario).

Also, I remember two cases where we got an issue like this before, and both were because the input we were giving to Prosimos was in the wrong format.