setup.py on Ubuntu20.04: tensorflow is wrong version about calamari

[jbarth-ubhd]

same within venv:

> . ~/calamari_venv/bin/activate
(calamari_venv) jb@pers16:~/calamari> python3 setup.py install
...
Installing tqdm-4.64.1-py2.py3-none-any.whl to /home/jb/calamari_venv/lib/python3.8/site-packages
Adding tqdm 4.64.1 to easy-install.pth file
Installing tqdm script to /home/jb/calamari_venv/bin

Installed /home/jb/calamari_venv/lib/python3.8/site-packages/tqdm-4.64.1-py3.8.egg
error: tensorflow 2.12.0rc0 is installed but tensorflow<2.7.0,>=2.4.0 is required by {'tfaip'}

[andbue]

Do you know where tensorflow 2.12 in your venv came from? Maybe it helps to first pip install "tensorflow<=2.7"?

[jbarth-ubhd]

tried again:

jb@pers16:~/calamari> python3 -m venv ~/calamari_venv
jb@pers16:~/calamari> . ~/calamari_venv/bin/activate
(calamari_venv) jb@pers16:~/calamari> python3 setup.py install
(a lot of log output...)
Installed /home/jb/calamari_venv/lib/python3.8/site-packages/tfaip-1.2.6-py3.8.egg
Searching for tensorflow>=2.4.0
Reading https://pypi.org/simple/tensorflow/
Downloading https://files.pythonhosted.org/packages/04/6c/
►89ca9fa1c0bd8500be17836aa1a4fc14b605cada5a97f83eda45bb7ee063/tensorflow-2.12.0rc0 ...
...
Installed /home/jb/calamari_venv/lib/python3.8/site-packages/typeguard-3.0.0rc2-py3.8.egg
Searching for tqdm
Reading https://pypi.org/simple/tqdm/
Downloading https://files.pythonhosted.org/packages/47/bb/
►849011636c4da2e44f1253cd927cfb20ada4374d8b3a4e425416e84900cc/tqdm-4.64.1-py2.py3-none-any.whl#sha256=
►6fee160d6ffcd1b1c68c65f14c829c22832bc401726335ce92c52d395944a6a1
Best match: tqdm 4.64.1
Processing tqdm-4.64.1-py2.py3-none-any.whl
Installing tqdm-4.64.1-py2.py3-none-any.whl to /home/jb/calamari_venv/lib/python3.8/site-packages
Adding tqdm 4.64.1 to easy-install.pth file
Installing tqdm script to /home/jb/calamari_venv/bin
Installed /home/jb/calamari_venv/lib/python3.8/site-packages/tqdm-4.64.1-py3.8.egg
error: tensorflow 2.12.0rc0 is installed but tensorflow<2.7.0,>=2.4.0 is required by {'tfaip'}


(calamari_venv) jb@pers16:~/calamari> pip3 install "tensorflow<=2.7"
...
Collecting pyasn1>=0.1.3
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
ERROR: tfaip 1.2.6 requires adabelief-tf, which is not installed.
ERROR: tfaip 1.2.6 requires dataclasses-json, which is not installed.
ERROR: tfaip 1.2.6 requires GitPython, which is not installed.
ERROR: tfaip 1.2.6 requires imageio, which is not installed.
ERROR: tfaip 1.2.6 requires nptyping, which is not installed.
ERROR: tfaip 1.2.6 requires opencv-python-headless, which is not installed.
ERROR: tfaip 1.2.6 requires openpyxl, which is not installed.
ERROR: tfaip 1.2.6 requires paiargparse==1.1.2, which is not installed.
ERROR: tfaip 1.2.6 requires pandas, which is not installed.
ERROR: tfaip 1.2.6 requires pillow, which is not installed.
ERROR: tfaip 1.2.6 requires pooch==1.4.0, which is not installed.
ERROR: tfaip 1.2.6 requires prettytable, which is not installed.
ERROR: tfaip 1.2.6 requires python-Levenshtein, which is not installed.
ERROR: tfaip 1.2.6 requires scikit-image, which is not installed.
ERROR: tfaip 1.2.6 requires tensorflow-addons>=0.12.0, which is not installed.
ERROR: tfaip 1.2.6 has requirement tensorflow<2.7.0,>=2.4.0, but you'll have tensorflow 2.7.0 which is 
►incompatible.
Installing collected packages: grpcio, wrapt, absl-py, tensorflow-io-gcs-filesystem, typing-extensions, termcolor
►, six, numpy, keras-preprocessing, opt-einsum, keras, flatbuffers, google-pasta, gast, wheel, astunparse, protobuf
►, tensorflow-estimator, libclang, MarkupSafe, werkzeug, tensorboard-data-server, zipp, importlib-metadata, 
►markdown, tensorboard-plugin-wit, certifi, urllib3, idna, charset-normalizer, requests, oauthlib, 
►requests-oauthlib, cachetools, pyasn1, rsa, pyasn1-modules, google-auth, google-auth-oauthlib, tensorboard, 
►tensorflow
  Attempting uninstall: tensorflow
    Found existing installation: tensorflow 2.12.0rc0
    Uninstalling tensorflow-2.12.0rc0:
      Successfully uninstalled tensorflow-2.12.0rc0
Successfully installed MarkupSafe-2.1.2 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.0 certifi-2022.12.7 
►charset-normalizer-3.0.1 flatbuffers-2.0.7 gast-0.4.0 google-auth-2.16.2 google-auth-oauthlib-0.4.6 
►google-pasta-0.2.0 grpcio-1.51.3 idna-3.4 importlib-metadata-6.0.0 keras-2.7.0 keras-preprocessing-1.1.2 
►libclang-15.0.6.1 markdown-3.4.1 numpy-1.24.2 oauthlib-3.2.2 opt-einsum-3.3.0 protobuf-4.22.0 pyasn1-0.4.8 
►pyasn1-modules-0.2.8 requests-2.28.2 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tensorboard-2.12.0 
►tensorboard-data-server-0.7.0 tensorboard-plugin-wit-1.8.1 tensorflow-2.7.0 tensorflow-estimator-2.7.0 
►tensorflow-io-gcs-filesystem-0.31.0 termcolor-2.2.0 typing-extensions-4.5.0 urllib3-1.26.14 werkzeug-2.2.3 
►wheel-0.38.4 wrapt-1.15.0 zipp-3.15.0

[jbarth-ubhd]

(calamari_venv) jb@pers16:~> calamari-predict
Traceback (most recent call last):
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tensorflow 2.7.0 (/home/jb/calamari_venv/lib/python3.8/site-packages), 
►Requirement.parse('tensorflow<2.7.0,>=2.4.0'), {'tfaip'})
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/jb/calamari_venv/bin/calamari-predict", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in 
►_initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in 
►_build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tensorflow 2.7.0 (/home/jb/calamari_venv/lib/python3.8/site-packages), 
►Requirement.parse('tensorflow<2.7.0,>=2.4.0'), {'tfaip'})


(calamari_venv) jb@pers16:~> pip3 list|grep tensor
tensorboard                  2.12.0   
tensorboard-data-server      0.7.0    
tensorboard-plugin-wit       1.8.1    
tensorflow                   2.7.0    
tensorflow-estimator         2.7.0    
tensorflow-io-gcs-filesystem 0.31.0   

[andbue]

I'm totally confused why your system enforces tensorflow>=2.7, I don't think we ever had this before. Could you please test if it works if you delete the tensorflow requirement in calamari's requirements.txt?

[jbarth-ubhd]

Oops pip3 install tensorflow<=2.7.0 was an error. Should be <2.7.0

[jbarth-ubhd]

after fixing some incompatible packages:

  File "/home/jb/calamari_venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'numpy~=1.19.2' distribution was not found and is required by tensorflow

ok so numpy 1.19.2 (but not 1.20 (`~=´))

(calamari_venv) jb@pers16:~/calamari> pip3 install numpy==1.19.2
Collecting numpy==1.19.2
  Using cached numpy-1.19.2-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
ERROR: tensorflow 2.6.5 has requirement typing-extensions<3.11,>=3.7, but you'll have typing-extensions 4.5.0 which is incompatible.
ERROR: scikit-image 0.20.0 has requirement numpy>=1.21.1, but you'll have numpy 1.19.2 which is incompatible.
ERROR: pandas 1.5.3 has requirement numpy>=1.20.3; python_version < "3.10", but you'll have numpy 1.19.2 which is incompatible.
ERROR: nptyping 2.5.0 has requirement numpy<2.0.0,>=1.20.0; python_version >= "3.8", but you'll have numpy 1.19.2 which is incompatible.
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.24.2
    Uninstalling numpy-1.24.2:
      Successfully uninstalled numpy-1.24.2
Successfully installed numpy-1.19.2

(calamari_venv) jb@pers16:~/calamari> pip3 install 'typing-extensions<3.11'
Collecting typing-extensions<3.11
  Using cached typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
ERROR: nptyping 2.5.0 has requirement numpy<2.0.0,>=1.20.0; python_version >= "3.8", but you'll have numpy 1.19.2 which is incompatible.
ERROR: nptyping 2.5.0 has requirement typing-extensions<5.0.0,>=4.0.0; python_version < "3.10", but you'll have typing-extensions 3.10.0.2 which is incompatible.
ERROR: typeguard 3.0.0rc2 has requirement typing-extensions>=4.4.0, but you'll have typing-extensions 3.10.0.2 which is incompatible.
Installing collected packages: typing-extensions
  Attempting uninstall: typing-extensions
    Found existing installation: typing-extensions 4.5.0
    Uninstalling typing-extensions-4.5.0:
      Successfully uninstalled typing-extensions-4.5.0
Successfully installed typing-extensions-3.10.0.2

ok so numpy must be ~=1.19.2 (but not 1.20), but after typing-extensions it must be >=1.20.0