add_final and repeat in replay about rvt

[imankgoyal]

Hi @pipixiaqishi1,

Thanks for your interest in our work.

I found that each episode has a replay with ternimal=-1, which has the same value as the replay before it with ternimal=True. Is this necessary or out of any special consideration?

I am unsure what you are referring to. Could you point me the code? Also, I am assuming this is something we borrowed from PerAct (https://github.com/peract/peract) to keep the dataset same.

And also there are many repeated replays generated by the loop
Yes this is a valid point. This design choice was made to be consistent with PerAct, so that we can compare the models fairly. More information about it can be found here: https://github.com/nvlabs/rvt#q-what-resources-are-required-to-train-rvt

Best,
Ankit

[pipixiaqishi1]

Hi @imankgoyal,
Thanks for your reply! I got it about this

This design choice was made to be consistent with PerAct, so that we can compare the models fairly.

For the first question, the final keypoint is added on purpose with terminal=-1. I assumed this was the last keypoint, but I found the terminal=True in the last keypoint in the previous loop which confused me.

RVT/rvt/utils/dataset.py

Lines 313 to 324 in 0b170d7

	obs_dict_tp1 = extract_obs(
	obs_tp1,
	CAMERAS,
	t=k + 1 - next_keypoint_idx,
	prev_action=prev_action,
	episode_length=25,
	)
	obs_dict_tp1["lang_goal_embs"] = lang_embs[0].float().detach().cpu().numpy()
	obs_dict_tp1.pop("wrist_world_to_cam", None)
	obs_dict_tp1.update(final_obs)
	replay.add_final(task, task_replay_storage_folder, **obs_dict_tp1)

Also, it seems that the 'final' keypoint is not considered valid when sampling.
https://github.com/NVlabs/YARR/blob/72c37e6d31ff1b1d7ce131be5c8f80f66ee271d8/yarr/replay_buffer/uniform_replay_buffer.py#L637-L639

[imankgoyal]

Hi @pipixiaqishi1,

I see your confusion now. Unfortunately, I am unsure about that design choice as it was borrowed from peract (https://github.com/peract/peract/blob/b73cf504567122f81912964943a16cd83920e4d9/agents/peract_bc/launch_utils.py#L147-L209) and we wanted to keep the dataset same for fairness.

Regardless, as you said, the 'final' keypoint is not considered for sampling so this might not eventaully cause any issue.

Best,
Ankit

[pipixiaqishi1]

Hi @imankgoyal,

Thanks very much for your patient response. It's okay to close this issue.

Best,
Jerry