AttributeError: module 'gentle' has no attribute 'Resources' about gently

[BayesForDays]

Hi Youseff, I've been working on a port that will make this more usable and to replicate the environment. I have also had this problem moving everything to a new virtual environment.

Your message has given me new motivation to figure out what is wrong and why it works in one of my environments and not another. I'll get back to you ASAP.

[youssefavx]

Hey Cassandra,

Awesome! Thanks so much and no pressure whatsoever. I wonder if gently also allows the production of sentence and paragraph or line by line start and end time stamps. I got the Gentle app to work but I’ve been having quite a difficult time working with the Gentle align csv output to transform it from a timestamp per word to a timestamp per line. I tried parsing it but I really suck at that so I couldn’t figure out a way to do it. No worries if not though, appreciate all of what you’ve done with gently so far!

[BayesForDays]

Interesting application! I think it could in principle -- I don't know if you've tried using the Gentle standalone app, but I've definitely used it to transcribe multiple sentences before. The challenge is that transcription takes an exponential amount of time (if I recall correctly) with the length of the audio file.

But if we assume you have the word-by-word alignments, the harder alignment problem is figuring out whether a sequence of transcriptions is a full sentence or not. You could in principle loop over the words in the transcription and terminate when you get to the end of a sentence i and the start of the next sentence i+1.

I wonder if a more efficient mechanism would be some kind of greedy sentence-splitting procedure...or if you know if there's a good chance that an utterance will take about a certain amount of time, you could chop the file into pieces and align in segments maybe.

[BayesForDays]

By the way, I just realized that while the package requires the gentle python package, it's also important to install Kaldi and the other dependencies. I got this error and am trying to work it out:

No resource directory .../gently/gentle/exp. Check GENTLE_RESOURCES_ROOT environment variable?

Some searching through the Issues on the gentle repo led me to this: lowerquality/gentle#168

If you could try the following commands, I think we could figure out if it's a dependency problem with gently calling the right resources:

git clone https://github.com/lowerquality/gentle.git
cd gentle
./install.sh

It does take a while to install Kaldi (which is a C program), so I don't know if this will work, but I ran into a problem where I was missing resources as well

I don't think it depends on where you clone gentle to, but it might. I'm currently trying it from withing the gently main directory.

[BayesForDays]

Tiny update: I have installed gentle (using the install.sh script provided by that package), but now I have found a bug in my textgrid code. See this issue for ongoing updates on that. I will update this issue as well once I have confirmed that the install procedure for gentle/Kaldi works.

[youssefavx]

Yeah I tried using the standalone app. Oh sorry, I forgot to say I already have the transcript for the audio (manually transcribed) the trouble comes when trying to get the starting timestamp for every paragraph in the transcript text file.

What I tried was to create an index for all the words that were isolated in the CSV file output by Gentle. Then, to do a paragraph start and end timestamp, I’d basically loop over the transcript text file line by line, and then check if every word in the align csv file is in that line, and if it is, add the index of that word to a dictionary like so: {line_index:[word_index, word_index, etc.]} and then I'd check if the word index was already in one of the values' arrays in that dictionary and if not, I'd add it. Unfortunately, this just gave me a mess of numbers. The word indexes would start at a certain point then you'd get words that aren't even in that line for some reason (or maybe they are, but I imagine they're probably 'the' or some other common word from another paragraph or something. So I think the problem area is that line 'if word in line' because it searches through the entire list of words as opposed to ignore previously looked at words. Anyway, I'd rather just throw away this code snippet or something cause it's the weirdest problem I've ever had to deal with.

This is the awful code:

aligndict = {}
with open('align with index.csv', 'r') as f:
    csv_reader = csv.reader(f, delimiter=',')
    for row in csv_reader:
        index = row[0]
        start = row[3]
        end = row[4]
        word = row[1]
        aligndict.update({index:[word, start, end]})


with open('transcript.txt', 'r') as f:
    lines = f.readlines()

lines_dict = {}
for index_for_line, line in enumerate(lines):
    lines_dict.update({index_for_line:line})

lineindexwordindex = {}
for index, wordstartend in aligndict.items():
    word = wordstartend[0]
    for lineindex, line in lines_dict.items():
        if word in line:
            if lineindex not in lineindexwordindex.keys():
                if not any(index in sublist for sublist in lineindexwordindex.values()):
                    lineindexwordindex.update({lineindex:[index]})
            elif lineindex in lineindexwordindex.keys():
                if not any(index in sublist for sublist in lineindexwordindex.values()):
                    lineindexwordindex[lineindex].append(index)

print(lineindexwordindex)

Anyway sorry to bore you with this.

I tried the commands you provided and cloned it to my home user folder, it installed, then I tried running the gently code again and got the same error unfortunately. I've placed the 'gently' folder in my Documents directory (which is different from where I installed Gentle) so I could try placing it in my user home folder and running the code there too if that has an effect.

Traceback (most recent call last):
  File "gentlyaligner.py", line 1, in <module>
    from gently.aligner import align_file, alignment_to_textgrid, save_textgrid
  File "/Users/josephvanrowe/Documents/gently/gently/aligner.py", line 13, in <module>
    resources = gentle.Resources()
AttributeError: module 'gentle' has no attribute 'Resources'

However, I must emphasize, that when I run, I run with python3.6 because gently failed to install on python 3.7. So I'm wondering if Gentle or aspects of Gentle or Kali may be installed in python3.7 when they're really needed in python3.6.

Here is what happens when I try to install gently in python3.7.2:

Obtaining file:///Users/josephvanrowe/Documents/gently
Requirement already satisfied: praatio in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from gently==0.0.1) (4.0.0)
Requirement already satisfied: pandas in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from gently==0.0.1) (0.25.1)
Requirement already satisfied: wave in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from gently==0.0.1) (0.0.2)
Processing /Users/josephvanrowe/Library/Caches/pip/wheels/7c/10/0a/87cc34721fe377bccd19d5d0525f25989765c02e1109a2f1af/Gentle-0.1-py3-none-any.whl
Requirement already satisfied: numpy>=1.13.3 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas->gently==0.0.1) (1.18.1)
Requirement already satisfied: python-dateutil>=2.6.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas->gently==0.0.1) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas->gently==0.0.1) (2019.1)
Collecting fabric
  Using cached fabric-2.5.0-py2.py3-none-any.whl (51 kB)
Requirement already satisfied: docopt in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from gentle->gently==0.0.1) (0.6.2)
Requirement already satisfied: six>=1.5 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas->gently==0.0.1) (1.12.0)
Requirement already satisfied: paramiko>=2.4 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from fabric->gentle->gently==0.0.1) (2.7.1)
Collecting invoke<2.0,>=1.3
  Using cached invoke-1.4.1-py3-none-any.whl (210 kB)
Requirement already satisfied: bcrypt>=3.1.3 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from paramiko>=2.4->fabric->gentle->gently==0.0.1) (3.1.7)
Requirement already satisfied: cryptography>=2.5 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from paramiko>=2.4->fabric->gentle->gently==0.0.1) (2.7)
Requirement already satisfied: pynacl>=1.0.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from paramiko>=2.4->fabric->gentle->gently==0.0.1) (1.3.0)
Requirement already satisfied: cffi>=1.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko>=2.4->fabric->gentle->gently==0.0.1) (1.12.3)
Requirement already satisfied: asn1crypto>=0.21.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from cryptography>=2.5->paramiko>=2.4->fabric->gentle->gently==0.0.1) (0.24.0)
Requirement already satisfied: pycparser in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko>=2.4->fabric->gentle->gently==0.0.1) (2.19)
Installing collected packages: invoke, fabric, gentle, gently
ERROR: Exception:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/py_compile.py", line 143, in compile
    _optimize=optimize)
  File "<frozen importlib._bootstrap_external>", line 791, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/private/var/folders/g0/5zwy4mtx7579v5x6rxqb083r0000gn/T/pip-unpacked-wheel-u583kdlg/invoke/vendor/yaml2/constructor.py", line 130
    except TypeError, exc:
                    ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/compileall.py", line 159, in compile_file
    invalidation_mode=invalidation_mode)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/py_compile.py", line 147, in compile
    raise py_exc
py_compile.PyCompileError:   File "/private/var/folders/g0/5zwy4mtx7579v5x6rxqb083r0000gn/T/pip-unpacked-wheel-u583kdlg/invoke/vendor/yaml2/constructor.py", line 130
    except TypeError, exc:
                    ^
SyntaxError: invalid syntax


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 404, in run
    use_user_site=options.use_user_site,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/req/__init__.py", line 71, in install_given_reqs
    **kwargs
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 815, in install
    warn_script_location=warn_script_location,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/operations/install/wheel.py", line 614, in install_wheel
    warn_script_location=warn_script_location,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/operations/install/wheel.py", line 338, in install_unpacked_wheel
    compileall.compile_dir(source, force=True, quiet=True)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/compileall.py", line 97, in compile_dir
    legacy, optimize, invalidation_mode):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/compileall.py", line 169, in compile_file
    msg = err.msg.encode(sys.stdout.encoding,
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 554, in encoding
    return self.orig_stream.encoding
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py", line 409, in __getattr__
    return getattr(self.stream, name)
AttributeError: '_io.BufferedWriter' object has no attribute 'encoding'

[youssefavx]

Hey Cassandra! So I have been trying to get gentle to work again with a python script, and unfortunately failing to do so. In the process, I tried installing gentle and had more problems with it. Then I said let me try gently again, and surprisingly got a different error this time:

  File "gentlyaligner.py", line 1, in <module>
    from gently.aligner import align_file, alignment_to_textgrid, save_textgrid
  File "/Users/josephvanrowe/Documents/gently/gently/aligner.py", line 7, in <module>
    import gentle
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gentle/__init__.py", line 2, in <module>
    from .resources import Resources
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/gentle/resources.py", line 4, in <module>
    from .util.paths import get_resource, ENV_VAR
ModuleNotFoundError: No module named 'gentle.util'

This is different from the error I got in the past which was:

AttributeError: module 'gentle' has no attribute 'Resources'

The error I got last time looks awfully similar to the current error I'm getting from gentle, but not the one from gently.

Appreciate all the help so far, no need to reply if you're busy!

[BayesForDays]

Hi again! Thanks for following up. Did you go through all the installation steps for gentle? I found that I had to download the models and everything. Try these steps:

git clone https://github.com/lowerquality/gentle.git

cd gentle/

set -e

git submodule init
git submodule update

./install_deps.sh
(cd ext && ./install_kaldi.sh)
VERSION="0.03"
download_models() {
	local version="$1"
	local filename="kaldi-models-$version.zip"
	local url="https://lowerquality.com/gentle/$filename"
	echo $filename
	wget -O $filename $url --no-check-certificate
	unzip $filename
	rm $filename
}
echo "Downloading models for v$VERSION..." 1>&2
download_models $VERSION
cd ext && make depend && make

And then reinstall gentle, then gently?

Unrelatedly I am planning on porting the alignment model over to pykaldi (and dropping gentle as the backbone, stay tuned for that)

[youssefavx]

Thank you so much! This seems to include much more than on the gentle github.

All steps seemed to work fine until this one:

(cd ext && ./install_kaldi.sh)

I got this error:

 OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)

  OS               ... Darwin             
  Architecture     ... x86_64               
  BINARY           ... 64bit                 
  C compiler       ... CLANG  (cmd & version : Apple LLVM version 10.0.1 (clang-1001.0.46.4))
  Fortran compiler ... GFORTRAN  (cmd & version : GNU Fortran (Homebrew GCC 9.1.0) 9.1.0)
-n   Library Name     ... libopenblas_haswell-r0.3.9.dev.a
 (Single-threading)  
WARNING: If you plan to use the dynamic library libopenblas_haswell-r0.3.9.dev.dylib, you must run:

"make PREFIX=/your_installation_path/ install".

(or set PREFIX in Makefile.rule and run make install.
If you want to move the .dylib to a new location later, make sure you change
the internal name of the dylib with:

install_name_tool -id /new/absolute/path/to/libopenblas_haswell-r0.3.9.dev.dylib libopenblas_haswell-r0.3.9.dev.dylib

To install the library, you can run "make PREFIX=/path/to/your/installation install".

/Library/Developer/CommandLineTools/usr/bin/make -j 8 -f Makefile.install install
Generating openblas_config.h in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/include
Generating f77blas.h in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/include
Generating cblas.h in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/include
Copying LAPACKE header files to /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/include
Copying the static library to /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/lib
Copying the shared library to /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/lib
Generating openblas.pc in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/lib/pkgconfig
Generating OpenBLASConfig.cmake in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Generating OpenBLASConfigVersion.cmake in /Users/josephvanrowe/Documents/gagain/gentle/ext/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Install OK!
Configuring KALDI to use OPENBLAS
Configuring ...
Checking compiler g++ ...
Checking OpenFst library in  ...
***configure failed: Could not find file /include/fst/fst.h:
  you may not have installed OpenFst. See ../tools/INSTALL ***
Makefile:30: kaldi.mk: No such file or directory
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1

Should I continue or do something else?

[prlabu]

Hi all - reading through this and have had a few issues described here, including

Checking OpenFst library in  ...
***configure failed: Could not find file /include/fst/fst.h:
  you may not have installed OpenFst. See ../tools/INSTALL ***
Makefile:30: kaldi.mk: No such file or directory
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1

and

No resource directory .../gently/gentle/exp. Check GENTLE_RESOURCES_ROOT environment variable?

I would appreciate any updates you all have. I tried to solve the OpenFST issue with the solution proposed at lowerquality/gentle#257, but no luck.

My main issue has just been installing Kaldi (more specifically, installing the version required by gentle).