czlwang / ltl-environment-dev

LTL Environment Dev

Environment grid world for Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas, based on the Craft-world from Andreas et al. (2017)

Structure

• algo/ contains the code to train a model using Advantage Actor Critic
• language/ contains the code to generate a dataset of (sentence, formula, environment) examples
  • usage: language/dataset.py --n_sentence 10 --dataset_path examples_out
• worlds/craft_world.py contains all the rules of the craft world environment as well as it's GUI
  • usage: worlds/craft_world.py to obtain an GUI
• spot2ba.py contains the hooks into Spot needed to maintain the FSA
• ltl2tree.py contains the code that specifies the tree structure of the planner model. This tree is formed according to the parse of the given LTL formula.

ltl-environment-dev's Issues

Pre-trained model missing and replacement model shape mismatch

I'm running your code and loading the 'planner_path: "ltl/craft_new_best_0708.pt' file didn't exist. SO we swtiched to use planner_path: "ltl/craft_new_best_0708.pt". It's getting the following issues

Traceback (most recent call last):
  File "./reinforce.py", line 900, in <module>
    train(model, adam_optim, 
  File "./reinforce.py", line 611, in train
    loss = run_epoch(train_iter, 
  File "./reinforce.py", line 399, in run_epoch
    loss = forward_pass(batch, model,
  File "./reinforce.py", line 361, in forward_pass
    rewards = compute_ltl_rewards(mr_words, mr_lengths,
  File "/home/arc/grounded_LTL_parser/rewards.py", line 424, in compute_ltl_rewards
    agent.actor_critic.load_state_dict(torch.load(model_path, map_location=ltl_args.device)[0])#TODO location
  File "/home/arc/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1671, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LTLActorCritic:
	Unexpected key(s) in state_dict: "base.boundary.obs_linear.weight", "base.boundary.obs_linear.bias", "base.boundary.rnn.weight_ih_l0", "base.boundary.rnn.weight_hh_l0", "base.boundary.rnn.bias_ih_l0", "base.boundary.rnn.bias_hh_l0", "base.boundary.rnn.weight_ih_l1", "base.boundary.rnn.weight_hh_l1", "base.boundary.rnn.bias_ih_l1", "base.boundary.rnn.bias_hh_l1", "base.boundary.out_linear.weight", "base.boundary.out_linear.bias". 
	size mismatch for base.tree.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.workbench.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.factory.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.iron.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.gold.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.gem.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.C.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.&.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.|.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.G.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.F.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.X.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.U.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).
	size mismatch for base.!.obs_linear.weight: copying a param with shape torch.Size([64, 393]) from checkpoint, the shape in current model is torch.Size([64, 367]).