Validation steps sometimes do not align in the multiagent about pointblank

[rich-iannone]

Great work here! I kind of think that environment-insensitive hashing should be the new default. Combined with version info in the hash (great idea by the way), I don’t think users would run into too many problems with these changes.

We should also talk about the next release!

[yjunechoe]

I had a chance to dig into this a bit more, and I realize that there might also be a bug in test-get_multiagent_report.R. On main, the multiagent with 1st and 3rd agent returns:

create_multiagent(agent_1, agent_3) %>%
  get_multiagent_report(display_mode = "wide")

With environment-insensitive hashing, we get 2 additional steps aligned:

As I suspected, the newly aligned steps are those involving vars() in values/left/right:

# Steps 5 and 6 of agent 1
col_vals_equal(vars(d), vars(d), na_pass = TRUE)
col_vals_between(vars(c), left = vars(a), right = vars(d), na_pass = TRUE)

# Steps 4 and 6 of agent 3
col_vals_equal(
  vars(d), vars(d),
  na_pass = TRUE
)
col_vals_between(
  vars(c),
  left = vars(a), right = vars(d),
  na_pass = TRUE
)

I think this is a bug but just wanted to get a second pair of eyes to confirm this.

---

Relatedly: should we introduce environment-insensitive hashing as an opt-in alternative hashing method, or should that become the new default for future agents?

Regardless of what we decide on above, it should be trivial to re-hash pre-existing agents for correct alignment since x_write_disk() saves out the entire $validation_set . On this note, I think it'd also be nice if we could append the current pkg version to the hash every time we make changes to the hashing implementations. So every hash created with this new env-insensitive hashing can end with _v0.12 (or whatever the next pointblank release would be), and we can use that to automatically trigger re-hashing of steps across agents if they span across different hashing implementations.

[yjunechoe]

Thanks! Will go ahead with that in the linked PR. And, yes - let me know what we'd need for the release.