qxcv / asnets

Source for Action Schema Networks paper (AAAI'18)

Implementation of ASNets

Update August 2019: This repository has been updated with a new copy of the ASNets code. The new code is substantially cleaner and includes new functionality described in a journal article that is currently under review. The old code and saved models for reproducing the AAAI paper are still available in the aaai18 branch.

This repository contains the code used in the AAAI'18 paper Action Schema Networks: Generalised Policies with Deep Learning, as well as a follow-up journal article that is currently under review. The abstract of the AAAI paper explains the idea:

In this paper, we introduce the Action Schema Network (ASNet): a neural network architecture for learning generalised policies for probabilistic planning problems. By mimicking the relational structure of planning problems, ASNets are able to adopt a weight-sharing scheme which allows the network to be applied to any problem from a given planning domain. This allows the cost of training the network to be amortised over all problems in that domain. Further, we propose a training method which balances exploration and supervised training on small problems to produce a policy which remains robust when evaluated on larger problems. In experiments, we show that ASNet's learning capability allows it to significantly outperform traditional non-learning planners in several challenging domains.

A longer, informal explanation can be found in a blog post on ASNets that discusses the AAAI'18 submission.

This repository is structured as follows:

• asnets/ contains our implementation and experiment files. Consult asnets/README.md for instructions on installing and running the code.
• problems/ includes all problems that we used to train + test the network, plus some problems which might be helpful for further research or debugging.
• materials/ contains papers and slides relevant to ASNets. They may be helpful for understanding this code, or ASNets more generally.

If you use this code in an academic publication, we'd appreciate it if you cited the following paper:

@inproceedings{toyer2018action,
  title={Action Schema Networks: Generalised Policies with Deep Learning},
  author={Toyer, Sam and Trevizan, Felipe and Thi{\'e}baux, Sylvie and Xie, Lexing},
  booktitle={AAAI Conference on Artificial Intelligence (AAAI)},
  year={2018}
}

Comments & queries can go to Sam Toyer.

asnets's Issues

Error

When running docker build on the DOCKERFILE, I end up getting this error:

ERROR: Command errored out with exit status 1:
 command: /home/asnets/venv-asnets/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/asnets/asnets/setup.py'"'"'; __file__='"'"'/home/asnets/asnets/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
     cwd: /home/asnets/asnets/
Complete output (242 lines):
running develop
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -std=c++11 -march=native -Ofast -I/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include -D_GLIBCXX_USE_CXX11_ABI=1 -g -I/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include -c asnets/ops/_asnet_ops_impl.cc -o asnets/ops/_asnet_ops_impl.o
In file included from /home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/unsupported/Eigen/CXX11/Tensor:140,
                 from /home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from /home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/numeric_types.h:20,
                 from /home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/allocator.h:23,
                 from /home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23,
                 from asnets/ops/_asnet_ops_impl.cc:1:
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorChipping.h: In member function ‘void Eigen::TensorEvaluator<const Eigen::TensorChippingOp<DimId, XprType>, Device>::block(Eigen::TensorEvaluator<const Eigen::TensorChippingOp<DimId, XprType>, Device>::OutputTensorBlock*) const [with long int DimId = -1; ArgType = const Eigen::TensorChippingOp<-1, const Eigen::TensorMap<Eigen::Tensor<const float, 3, 1, long int>, 16, Eigen::MakePointer> >; Device = Eigen::DefaultDevice]’:
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorChipping.h:317:9: warning: ‘*((void*)(& input_block)+8)[18446744073709551615]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  317 |         input_block_sizes[i] = output_block_sizes[i - 1];
      |         ^~~~~~~~~~~~~~~~~
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorChipping.h: In member function ‘void Eigen::TensorEvaluator<const Eigen::TensorCwiseBinaryOp<BinaryOp, LeftArgType, RightArgType>, Device>::block(Eigen::TensorEvaluator<const Eigen::TensorCwiseBinaryOp<BinaryOp, LeftArgType, RightArgType>, Device>::TensorBlock*) const [with BinaryOp = Eigen::internal::scalar_sum_op<float, float>; LeftArgType = const Eigen::TensorChippingOp<-1, Eigen::TensorChippingOp<-1, Eigen::TensorMap<Eigen::Tensor<float, 3, 1, long int>, 16, Eigen::MakePointer> > >; RightArgType = const Eigen::TensorSlicingOp<const std::array<long long int, 1>, const std::array<long long int, 1>, const Eigen::TensorChippingOp<0, const Eigen::TensorChippingOp<0, const Eigen::TensorMap<Eigen::Tensor<const float, 3, 1, long int>, 16, Eigen::MakePointer> > > >; Device = Eigen::DefaultDevice]’:
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/include/unsupported/Eigen/CXX11/src/Tensor/TensorChipping.h:317:9: warning: ‘*((void*)(& input_block)+8)[18446744073709551615]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  317 |         input_block_sizes[i] = output_block_sizes[i - 1];
      |         ^~~~~~~~~~~~~~~~~
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g asnets/ops/_asnet_ops_impl.o -o asnets/ops/_asnet_ops_impl.so -std=c++11 -march=native -Ofast -lstdc++ -Wl,--no-undefined -L/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow -ltensorflow_framework
running egg_info
creating asnets.egg-info
writing asnets.egg-info/PKG-INFO
writing dependency_links to asnets.egg-info/dependency_links.txt
writing requirements to asnets.egg-info/requires.txt
writing top-level names to asnets.egg-info/top_level.txt
writing manifest file 'asnets.egg-info/SOURCES.txt'
reading manifest file 'asnets.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'asnets.egg-info/SOURCES.txt'
running build_ext
Creating /home/asnets/venv-asnets/lib/python3.7/site-packages/asnets.egg-link (link to .)
Adding asnets 0.0.1 to easy-install.pth file

Installed /home/asnets/asnets
requesting all changes
adding changesets
adding manifests
adding file changes
added 13771 changesets with 40247 changes to 9187 files (+4 heads)
new changesets 875fe8fa322a:50e4b39850c3
updating to branch release-19.06
1467 files updated, 0 files merged, 0 files removed, 0 files unresolved
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building for 64-bit.
-- Performing Test CXX11_FOUND
-- Performing Test CXX11_FOUND - Success
-- Enabling plugin causal_graph because plugin core_sources is enabled and depends on it.
-- Enabling plugin int_hash_set because plugin core_sources is enabled and depends on it.
-- Enabling plugin int_packer because plugin core_sources is enabled and depends on it.
-- Enabling plugin ordered_set because plugin core_sources is enabled and depends on it.
-- Enabling plugin segmented_vector because plugin core_sources is enabled and depends on it.
-- Enabling plugin subscriber because plugin core_sources is enabled and depends on it.
-- Enabling plugin successor_generator because plugin core_sources is enabled and depends on it.
-- Enabling plugin task_properties because plugin core_sources is enabled and depends on it.
-- Enabling plugin combining_evaluator because plugin max_evaluator is enabled and depends on it.
-- Enabling plugin stubborn_sets because plugin stubborn_sets_simple is enabled and depends on it.
-- Enabling plugin eager_search because plugin plugin_astar is enabled and depends on it.
-- Enabling plugin search_common because plugin plugin_astar is enabled and depends on it.
-- Enabling plugin lazy_search because plugin plugin_lazy is enabled and depends on it.
-- Enabling plugin priority_queues because plugin additive_heuristic is enabled and depends on it.
-- Enabling plugin relaxation_heuristic because plugin additive_heuristic is enabled and depends on it.
-- Enabling plugin domain_transition_graph because plugin context_enhanced_additive_heuristic is enabled and depends on it.
-- Enabling plugin dynamic_bitset because plugin cegar is enabled and depends on it.
-- Enabling plugin extra_tasks because plugin cegar is enabled and depends on it.
-- Enabling plugin equivalence_relation because plugin mas_heuristic is enabled and depends on it.
-- Enabling plugin sccs because plugin mas_heuristic is enabled and depends on it.
-- Enabling plugin variable_order_finder because plugin mas_heuristic is enabled and depends on it.
-- Enabling plugin lp_solver because plugin landmarks is enabled and depends on it.
-- Enabling plugin max_cliques because plugin pdbs is enabled and depends on it.
-- Enabling plugin sampling because plugin pdbs is enabled and depends on it.
-- Enabling plugin null_pruning_method because plugin eager_search is enabled and depends on it.
-- Using plugin: core_sources
-- Using plugin: options
-- Using plugin: utils
-- Using plugin: alternation_open_list
-- Using plugin: best_first_open_list
-- Using plugin: epsilon_greedy_open_list
-- Using plugin: pareto_open_list
-- Using plugin: tiebreaking_open_list
-- Using plugin: type_based_open_list
-- Using plugin: dynamic_bitset
-- Using plugin: equivalence_relation
-- Using plugin: int_hash_set
-- Using plugin: int_packer
-- Using plugin: max_cliques
-- Using plugin: priority_queues
-- Using plugin: ordered_set
-- Using plugin: segmented_vector
-- Using plugin: subscriber
-- Using plugin: evaluators_plugin_group
-- Using plugin: const_evaluator
-- Using plugin: g_evaluator
-- Using plugin: combining_evaluator
-- Using plugin: max_evaluator
-- Using plugin: pref_evaluator
-- Using plugin: weighted_evaluator
-- Using plugin: sum_evaluator
-- Using plugin: null_pruning_method
-- Using plugin: stubborn_sets
-- Using plugin: stubborn_sets_simple
-- Using plugin: stubborn_sets_ec
-- Using plugin: search_common
-- Using plugin: eager_search
-- Using plugin: plugin_astar
-- Using plugin: plugin_eager
-- Using plugin: plugin_eager_greedy
-- Using plugin: plugin_eager_wastar
-- Using plugin: plugin_lazy
-- Using plugin: plugin_lazy_greedy
-- Using plugin: plugin_lazy_wastar
-- Using plugin: enforced_hill_climbing_search
-- Using plugin: iterated_search
-- Using plugin: lazy_search
-- Using plugin: lp_solver
-- Using plugin: relaxation_heuristic
-- Using plugin: additive_heuristic
-- Using plugin: blind_search_heuristic
-- Using plugin: context_enhanced_additive_heuristic
-- Using plugin: cg_heuristic
-- Using plugin: domain_transition_graph
-- Using plugin: ff_heuristic
-- Using plugin: goal_count_heuristic
-- Using plugin: hm_heuristic
-- Using plugin: landmark_cut_heuristic
-- Using plugin: max_heuristic
-- Using plugin: core_tasks
-- Using plugin: extra_tasks
-- Using plugin: causal_graph
-- Using plugin: sampling
-- Using plugin: successor_generator
-- Using plugin: task_properties
-- Using plugin: variable_order_finder
-- Using plugin: cegar
-- Using plugin: mas_heuristic
-- Using plugin: landmarks
-- Using plugin: operator_counting
-- Using plugin: pdbs
-- Using plugin: potentials
-- Using plugin: sccs
-- Could NOT find OSI (missing: OSI_INCLUDE_DIRS OSI_LIBRARIES)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/asnets/asnets/fd/builds/release
Scanning dependencies of target translate
Copying translator module into output directory
[  0%] Built target translate
Scanning dependencies of target downward
[  0%] Building CXX object search/CMakeFiles/downward.dir/algorithms/sccs.cc.o
[  0%] Building CXX object search/CMakeFiles/downward.dir/potentials/util.cc.o
[  1%] Building CXX object search/CMakeFiles/downward.dir/potentials/potential_optimizer.cc.o
[  1%] Building CXX object search/CMakeFiles/downward.dir/potentials/sample_based_potential_heuristics.cc.o
[  2%] Building CXX object search/CMakeFiles/downward.dir/potentials/single_potential_heuristics.cc.o
[  2%] Building CXX object search/CMakeFiles/downward.dir/potentials/potential_max_heuristic.cc.o
[  3%] Building CXX object search/CMakeFiles/downward.dir/potentials/potential_heuristic.cc.o
[  3%] Building CXX object search/CMakeFiles/downward.dir/potentials/potential_function.cc.o
[  3%] Building CXX object search/CMakeFiles/downward.dir/potentials/diverse_potential_heuristics.cc.o
[  4%] Building CXX object search/CMakeFiles/downward.dir/potentials/plugin_group.cc.o
[  5%] Building CXX object search/CMakeFiles/downward.dir/pdbs/zero_one_pdbs_heuristic.cc.o
[  5%] Building CXX object search/CMakeFiles/downward.dir/pdbs/zero_one_pdbs.cc.o
[  6%] Building CXX object search/CMakeFiles/downward.dir/pdbs/validation.cc.o
[  6%] Building CXX object search/CMakeFiles/downward.dir/pdbs/utils.cc.o
[  7%] Building CXX object search/CMakeFiles/downward.dir/pdbs/plugin_group.cc.o
[  7%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pdb_heuristic.cc.o
[  8%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_information.cc.o
[  8%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_generator.cc.o
[  9%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_generator_manual.cc.o
[  9%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_generator_greedy.cc.o
[ 10%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_database.cc.o
[ 10%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_generator_systematic.cc.o
[ 10%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_generator_manual.cc.o
[ 11%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_generator_hillclimbing.cc.o
[ 11%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_generator_genetic.cc.o
[ 12%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_generator_combo.cc.o
[ 12%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_collection_information.cc.o
[ 13%] Building CXX object search/CMakeFiles/downward.dir/pdbs/pattern_cliques.cc.o
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[2]: *** [search/CMakeFiles/downward.dir/build.make:180: search/CMakeFiles/downward.dir/potentials/diverse_potential_heuristics.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[2]: *** [search/CMakeFiles/downward.dir/build.make:102: search/CMakeFiles/downward.dir/potentials/sample_based_potential_heuristics.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:128: search/CMakeFiles/downward.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
Building configuration release.
Executing command "cmake -G Unix Makefiles -DCMAKE_BUILD_TYPE=Release ../../src" in directory "builds/release".
Executing command "make -j2 -j16" in directory "builds/release".
Traceback (most recent call last):
  File "build.py", line 153, in <module>
    main()
  File "build.py", line 149, in main
    build(config_name, CONFIGS[config_name], make_parameters)
  File "build.py", line 126, in build
    try_run([MAKE] + make_parameters, cwd=build_path)
  File "build.py", line 102, in try_run
    subprocess.check_call(cmd, cwd=cwd)
  File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j2', '-j16']' returned non-zero exit status 2.
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/asnets/venv-asnets/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/asnets/asnets/setup.py", line 133, in <module>
    'build_py': build_py_and_ext,
  File "/home/asnets/venv-asnets/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/asnets/asnets/setup.py", line 61, in run
    try_install_fd()
  File "/home/asnets/asnets/asnets/fd_interface.py", line 54, in try_install_fd
    cwd=FD_DIR)
  File "/usr/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['python', 'build.py', '-j16']' returned non-zero exit status 1.
Installing FD
----------------------------------------

ERROR: Command errored out with exit status 1: /home/asnets/venv-asnets/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/asnets/asnets/setup.py'"'"'; file='"'"'/home/asnets/asnets/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
The command '/bin/sh -c . ${HOME}/venv-asnets/bin/activate && echo . ${HOME}/venv-asnets/bin/activate >> ${HOME}/.bashrc && pip3 install --upgrade pip && pip3 install -e ${HOME}/asnets/' returned a non-zero code: 1

how to install the mdpsim module and ssipp module which appear in the supervised.py?

I download the code and try to run it but something go wrong.
The error is ModuleNotFoundError: No module name 'mdpsim'.
I check the code and find it appear in the line 189 and line 190 ,can anyone tell me how to solve it ?