louiskirsch / tensorflow-with-kenlm Goto Github PK

I am getting an error of segmentation fault(core dumped) while using kenlm within the ctc beam search decoder.

NOTE: Only file GitHub issues for bugs and feature requests. All other topics will be closed.

For general support from the community, see StackOverflow.
To make bugs and feature requests more easy to find and organize, we close issues that are deemed
out of scope for GitHub Issues and point people to StackOverflow.

For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

Environment info

Operating System:

Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*):

If installed from binary pip package, provide:

1. A link to the pip package you installed:
2. The output from python -c "import tensorflow; print(tensorflow.__version__)".

If installed from source, provide

1. The commit hash (git rev-parse HEAD)
2. The output of bazel version

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)

What other attempted solutions have you tried?

Logs or other output that would be helpful

(If logs are large, please upload as attachment or provide link).

Hi,

I ran ctc_beam_search_decoder and I found it too slow with kenlm.
My python code looks like below.

decoded, _ = tf.nn.ctc_beam_search_decoder(...)
with tf.Session() as session:
session.run(decoded) # Takes more than 20 seconds!

I profiled the code and found that it took most of the time at loading vocabulary file ('trie'). Is there any way that I can avoid repeated call to TrieNode::ReadFromStream for each session.run() ?
For now, it seems to called for each session.run(decoded) which seems quite redundant.

Thank you and best regards,
Hongjoo

The link for "Download and Setup" in the readme is broken. It leads to: https://github.com/timediv/tensorflow-with-kenlm/blob/master/tensorflow/g3doc/get_started/os_setup.md

This leads me to the error that when I clone and ./configure I get: no such package '@kenlm_archive//':

Hi @timediv , I stuck at bazel-bin/tensorflow/core/util/ctc/ctc_generate_trie kenlm-model.binary vocabulary < corpus.txt > trie it says: bash: corpus.txt: No such file or directory

So where can I find this corpus.txt file?

Also I did not understand the purpose that command. How am I suppose to upgrade my tensorflow installation to modify tf.nn.ctc_beam_search_decoder() function and support kenlm_directory_path='your/directory/path' argument?