davidgolub / simpleqa Goto Github PK

Creating a new input dssm layer for index 29 Creating new hidden network for dssm Sharing DSSM layer Sanity checking that dssm sharing worked ANSWER RERANK PREDICTING Topkargmax done ANSWER RERANK PREDICTING Topkargmax done Harder.....Faster /music/album/genre /m/06by7 /m/01qzt1 /m/0xhtw Harder.....Faster /music/album/genre /m/06by7 /m/01qzt1 /m/0xhtw Index Tot/Fact/Names/ pred/entity accuracy 1 1 0 0 1 1 Which genre of album is harder.....faster? /music/album/genre Harder.....Faster FACT:/m/06by7 /m/01qzt1 /m/0xhtw 2 /m/01jp8ww /root/torch/install/bin/luajit: scripts/evaluate_sq_fb_dataset_v2.lua:160: attempt to call field 'write' (a nil value) stack traceback: scripts/evaluate_sq_fb_dataset_v2.lua:160: in main chunk [C]: in function 'dofile' /root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00406670

hi, can I ask you to put subjects_names, objects_names, and all entities for the train and validation set too?

Hello, I followed your instructions to run the code. However I encountered the following problem. I'd like to know how to solve it~
Thanks a lot!

Creating a new input dssm layer for index 29 Creating new hidden network for dssm Sharing DSSM layer Sanity checking that dssm sharing worked ANSWER RERANK PREDICTING Topkargmax done ANSWER RERANK PREDICTING Topkargmax done Harder.....Faster /music/album/genre /m/06by7 /m/01qzt1 /m/0xhtw Harder.....Faster /music/album/genre /m/06by7 /m/01qzt1 /m/0xhtw Index Tot/Fact/Names/ pred/entity accuracy 1 1 0 0 1 1 Which genre of album is harder.....faster? /music/album/genre Harder.....Faster FACT:/m/06by7 /m/01qzt1 /m/0xhtw 2 /m/01jp8ww /root/torch/install/bin/luajit: scripts/evaluate_sq_fb_dataset_v2.lua:160: attempt to call field 'write' (a nil value) stack traceback: scripts/evaluate_sq_fb_dataset_v2.lua:160: in main chunk [C]: in function 'dofile' /root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00406670

When I run docker according to the steps in the readme, I encountered the following error. I am new to docker. And I didn't find a solution in Google search. Can you provide some help?
Thank you very much!

On 21000  
Reading raw line data from ../datasets/SimpleQuestions/test/object_names.txt  
On 10000  
On 20000` 
file (0x0355d060)
file (0x03584c60)
On index 1
Answering question Which genre of album is harder.....faster?
Getting all ids
genre album harder.....faster
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 257, in error_router
    return self.handle_error(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 257, in error_router
    return self.handle_error(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 397, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 487, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/src/servers/FreebaseWebServer/src/resource/freebase.py", line 31, in get
    names, num_items = fb_helper.get_names(removed_stopwords_query, num_results)
  File "/src/servers/FreebaseWebServer/src/util/freebase_helper.py", line 234, in get_names
    res = self.es.search(index=self.name_index, body=elastic_query)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 625, in search
    doc_type, '_search'), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 123, in perform_request
    raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe047610350>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe047610350>: Failed to establish a new connection: [Errno 111] Connection refused)
ERROR DECODING JSON /src/dmn/util/io_functions.lua:7: Expected value but found invalid token at character 1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
  "http://www.w3.org/TR/html4/loose.dtd">
<html>
  <head>
    <title>ConnectionError: ConnectionError(&lt;urllib3.connection.HTTPConnection object at 0x7fe047610350&gt;: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(&lt;urllib3.connection.HTTPConnection object at 0x7fe047610350&gt;: Failed to establish a new connection: [Errno 111] Connection refused) // Werkzeug Debugger</title>
    <link rel="stylesheet" href="?__debugger__=yes&amp;cmd=resource&amp;f=style.css" type="text/css">
    <!-- We need to make sure this has a favicon so that the debugger does not by
         accident trigger a request to /favicon.ico which might change the application
         state. -->
    <link rel="shortcut icon" href="?__debugger__=yes&amp;cmd=resource&amp;f=console.png">
    <script type="text/javascript" src="?__debugger__=yes&amp;cmd=resource&amp;f=jquery.js"></script>
    <script type="text/javascript" src="?__debugger__=yes&amp;cmd=resource&amp;f=debugger.js"></script>
    <script type="text/javascript">
      var TRACEBACK = 140601246941776,
          CONSOLE_MODE = false,
          EVALEX = true,
          SECRET = "pV3FF3oGSfrIe7zT0qHA";
    </script>
  </head>
  <body>
    <div class="debugger">
<h1>elasticsearch.exceptions.ConnectionError</h1> 

Error reported when building the docker，detailed infomation is as follow, it seems that the url is not avaliable now.
Thank you for your reply.

hongzhi@hongzhi-Runner:/mydata/wp/SimpleQA$ docker build -t simpleqa .
Sending build context to Docker daemon 42.4 MB
Step 1 : FROM ubuntu:14.04
---> 7c09e61e9035
Step 2 : RUN apt-get update
---> Using cache
---> b65fd1767de7
Step 3 : RUN apt-get install git -y
---> Using cache
---> ed320257ff4e
Step 4 : RUN apt-get install curl -y
---> Using cache
---> 63d3d1d00fb8
Step 5 : RUN apt-get install wget -y
---> Using cache
---> 58e1a803d795
Step 6 : RUN apt-get install python-dev -y
---> Using cache
---> 3c46882f8df4
Step 7 : RUN apt-get install python-pip -y
---> Using cache
---> fd3b26d0929b
Step 8 : RUN apt-get -y install python-pip
---> Using cache
---> 98203337c0d0
Step 9 : RUN pip install --upgrade numpy
---> Using cache
---> 143cb4a5aae1
Step 10 : RUN pip install cassandra-driver
---> Using cache
---> 836c5d37bf7a
Step 11 : RUN pip install cqlengine
---> Using cache
---> a0c3eae3668c
Step 12 : RUN pip install unidecode
---> Using cache
---> a5ee3761a6bb
Step 13 : RUN pip install elasticsearch
---> Using cache
---> faf22f6a1920
Step 14 : RUN apt-get install -y pv
---> Using cache
---> 37e1367c3225
Step 15 : RUN apt-get install -y libpq-dev
---> Using cache
---> f7e432a336ce
Step 16 : RUN pip install tinys3
---> Using cache
---> ae18e0806104
Step 17 : RUN pip install boto
---> Using cache
---> 5f1c6c5d7cef
Step 18 : RUN pip install filechunkio
---> Using cache
---> d91afa680678
Step 19 : RUN apt-get install -y software-properties-common
---> Using cache
---> 36abc9ca800e
Step 20 : RUN curl -sk https://raw.githubusercontent.com/davidgolub/WorkSamples/master/torch_install.sh install-deps | bash
---> Running in 297fddbdea0e
bash: line 1: 404:: command not found
The command '/bin/sh -c curl -sk https://raw.githubusercontent.com/davidgolub/WorkSamples/master/torch_install.sh install-deps | bash' returned a non-zero code: 127

Hello, thank you for your code.
But I'm very confused for the input vector in the LSTM-based decoder.
In this paper, "At time i = 2, during training, the input vector is the embedding of the true entity". But I find the input vector of the time i=2 is the true predicate in your code.

To my best knowledge, we can use Freebase2Wikidata mapping provided by Google to decode MIDs to strings.

e.g. www.freebase.com/m/0jtw9c

Is it the best way to preprocess SimpleQuestions dataset before training?

I tried to look at the lua code for read_sq_data but no clues came to my mind.

Thanks for your help. Great work!

Hello！Thank you so much for the data set! But where can the fact triples in the SimpleQuestion dataset be provided? I needed to do some work based on the original knowledge graph, but in my search, I didn't find anything relevant. If you know, please let me know. Thank you.