error when downloading dandiset on MATLAB instance about dandi-hub

[CodyCBakerPhD]

Note: this was specifically on the MATLAB DANDI Hub instance

(should probably be transferred to https://github.com/dandi/dandi-hub)

[bendichter]

oh, do you think that's part of the issue? That would explain why we haven't seen this error yet

[bendichter]

yup, this is specific to that instance. Thanks for catching that, @CodyCBakerPhD

[yarikoptic]

google leads me to https://levelup.gitconnected.com/fix-attributeerror-module-lib-has-no-attribute-openssl-521a35d83769 which outlines the issue and solution (python3 -m pip install --upgrade pip and/or python3 -m pip install openssl>22.1.0). @jwodder do we need to establish lower version for openssl ?

[jwodder]

@yarikoptic OpenSSL is not a PyPI package, so installing it via pip is not an option.

[yarikoptic]

d'oh -- so the article is wrong. It actually talks about pyopenssl , so may be python3 -m pip install 'pyopenssl>22.1.0'

[jwodder]

@bendichter Can you run the command again with the DANDI_DEVEL envvar set to something nonempty so we can see exactly where this error is occurring?

[bendichter]

@jwodder sure

Traceback (most recent call last):
  File "/opt/conda/bin/dandi", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/dandi/cli/base.py", line 128, in wrapper
    return f(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/dandi/cli/cmd_download.py", line 111, in download
    from .. import download
  File "/opt/conda/lib/python3.10/site-packages/dandi/download.py", line 44, in <module>
    from .dandiset import Dandiset
  File "/opt/conda/lib/python3.10/site-packages/dandi/dandiset.py", line 14, in <module>
    from .files import DandisetMetadataFile, LocalAsset, dandi_file, find_dandi_files
  File "/opt/conda/lib/python3.10/site-packages/dandi/files/__init__.py", line 23, in <module>
    from ._private import BIDSFileFactory, DandiFileFactory
  File "/opt/conda/lib/python3.10/site-packages/dandi/files/_private.py", line 17, in <module>
    from .bases import DandiFile, GenericAsset, LocalAsset, NWBAsset, VideoAsset
  File "/opt/conda/lib/python3.10/site-packages/dandi/files/bases.py", line 33, in <module>
    from dandi.support.digests import get_dandietag, get_digest
  File "/opt/conda/lib/python3.10/site-packages/dandi/support/digests.py", line 22, in <module>
    from zarr_checksum import ZarrChecksumTree
  File "/opt/conda/lib/python3.10/site-packages/zarr_checksum/__init__.py", line 4, in <module>
    from zarr_checksum.generators import FileGenerator
  File "/opt/conda/lib/python3.10/site-packages/zarr_checksum/generators.py", line 9, in <module>
    import boto3
  File "/opt/conda/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/opt/conda/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/opt/conda/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/opt/conda/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/opt/conda/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/opt/conda/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/opt/conda/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
  File "/opt/conda/lib/python3.10/site-packages/botocore/docs/client.py", line 14, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/opt/conda/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/opt/conda/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/opt/conda/lib/python3.10/site-packages/botocore/utils.py", line 34, in <module>
    import botocore.httpsession
  File "/opt/conda/lib/python3.10/site-packages/botocore/httpsession.py", line 41, in <module>
    from urllib3.contrib.pyopenssl import orig_util_SSLContext as SSLContext
  File "/opt/conda/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.SSL
  File "/opt/conda/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/opt/conda/lib/python3.10/site-packages/OpenSSL/crypto.py", line 3268, in <module>
    _lib.OpenSSL_add_all_algorithms()
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

[jwodder]

@bendichter What version of urllib3 does the environment have?

[bendichter]

@jwodder urllib3 is version 1.26.11

terminal activity

$ pip freeze
affine==2.4.0
aicsimageio==4.9.4
aiobotocore==2.4.2
aiohttp==3.8.4
aiohttp-session==2.12.0
aioitertools==0.11.0
aiosignal==1.3.1
alembic @ file:///home/conda/feedstock_root/build_artifacts/alembic_1657813896088/work
anndata==0.8.0
annexremote @ file:///home/conda/feedstock_root/build_artifacts/annexremote_1676779967743/work
anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1660053721269/work/dist
appdirs==1.4.4
argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1640817743617/work
argon2-cffi-bindings @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi-bindings_1649500328244/work
arrow==1.2.3
asciitree==0.3.3
astor==0.8.1
asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1660605382950/work
async-generator==1.10
async-timeout @ file:///home/conda/feedstock_root/build_artifacts/async-timeout_1640026696943/work
atomicwrites==1.4.1
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1659291887007/work
Babel @ file:///home/conda/feedstock_root/build_artifacts/babel_1655419414885/work
backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work
backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work
beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1649463573192/work
bids-validator==1.11.0
bidsschematools==0.7.1
bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1656355450470/work
blessings==1.7
blinker @ file:///home/conda/feedstock_root/build_artifacts/blinker_1664823096650/work
bokeh==3.1.0
boto==2.49.0
boto3==1.24.59
botocore==1.27.59
bqplot==0.12.36
Brotli==1.0.9
brotlipy @ file:///home/conda/feedstock_root/build_artifacts/brotlipy_1648854167867/work
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
cachetools==5.3.0
certifi==2022.12.7
certipy==0.1.3
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1656782834004/work
cfgrib==0.9.10.3
cftime==1.6.2
chardet @ file:///home/conda/feedstock_root/build_artifacts/chardet_1669990270610/work
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1661170624537/work
ci-info==0.3.0
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
cligj==0.7.2
cloud-files==4.12.1
cloud-volume==8.19.2
cloudpickle==2.2.1
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1655412516417/work
compressed-segmentation==2.2.0
compresso==3.2.0
conda==22.9.0
conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1663583648888/work
contourpy==1.0.7
crackle-codec==0.4.0
crc32c==2.3.post0
cryptography @ file:///home/conda/feedstock_root/build_artifacts/cryptography-split_1677821274828/work
cycler==0.11.0
dandi==0.55.1
dandiio @ https://github.com/balbasty/dandi-io/archive/refs/heads/main.zip
dandischema==0.8.2
dask==2023.3.1
datalad @ file:///home/conda/feedstock_root/build_artifacts/datalad_1677558998425/work
datalad-container==1.1.9
datalad-osf==0.2.3.1
debugpy @ file:///home/conda/feedstock_root/build_artifacts/debugpy_1660619028387/work
decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
deflate==0.3.0
defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work
Deprecated @ file:///home/conda/feedstock_root/build_artifacts/deprecated_1632758190537/work
dill==0.3.6
distro @ file:///home/conda/feedstock_root/build_artifacts/distro_1675116244235/work
dnspython==2.3.0
docopt==0.6.2
DracoPy==1.2.0
eccodes==1.5.1
elementpath==4.0.1
email-validator==1.3.1
entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
etelemetry==0.3.0
exceptiongroup==1.1.1
executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1665301981797/work
ExifRead @ file:///home/conda/feedstock_root/build_artifacts/exifread_1652071410078/work
fasteners @ file:///home/conda/feedstock_root/build_artifacts/fasteners_1643971550063/work
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1663619548554/work/dist
fastremap==1.13.4
filelock==3.10.0
findlibs==0.0.2
flit_core @ file:///home/conda/feedstock_root/build_artifacts/flit-core_1645629044586/work/source/flit_core
fonttools==4.39.2
formulaic==0.5.2
fpzip==1.2.0
fqdn==1.5.1
frozenlist==1.3.3
fscacher==0.3.0
fsspec==2023.3.0
future @ file:///home/conda/feedstock_root/build_artifacts/future_1673596611778/work
gast==0.4.0
gevent==22.10.2
google-api-core==2.11.0
google-apitools==0.5.32
google-auth==2.16.2
google-cloud-core==2.3.2
google-cloud-storage==2.7.0
google-crc32c==1.5.0
google-resumable-media==2.4.1
googleapis-common-protos==1.58.0
greenlet==2.0.2
h5netcdf==1.1.0
h5py @ file:///home/conda/feedstock_root/build_artifacts/h5py_1675704801301/work
hdf5plugin==4.1.1
hdmf==3.5.2
httplib2==0.21.0
humanize @ file:///home/conda/feedstock_root/build_artifacts/humanize_1675692848261/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1663625384323/work
imagecodecs==2023.3.16
imageio==2.26.1
importlib-metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1653252803161/work
importlib-resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1665204935269/work
inflection==0.5.1
iniconfig==2.0.0
interface-meta==1.3.0
interleave==0.2.1
ipydatagrid==1.1.15
ipydatawidgets==4.3.2
ipykernel @ file:///home/conda/feedstock_root/build_artifacts/ipykernel_1664214716573/work
ipympl==0.9.3
ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1662481517711/work
ipython-genutils==0.2.0
ipyvolume==0.6.1
ipyvue==1.9.0
ipyvuetify==1.8.4
ipywebrtc==0.6.0
ipywidgets==8.0.4
iso8601 @ file:///home/conda/feedstock_root/build_artifacts/iso8601_1664386709461/work
isodate==0.6.1
isoduration==20.11.0
jaraco.classes @ file:///home/conda/feedstock_root/build_artifacts/jaraco.classes_1667024629799/work
jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1659959867326/work
jeepney @ file:///home/conda/feedstock_root/build_artifacts/jeepney_1649085214306/work
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1654302431367/work
jmespath==1.0.1
joblib==1.2.0
json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1600692310011/work
jsonpointer==2.3
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema-meta_1662736858348/work
jupyter-bokeh==3.0.7
jupyter-contrib-core==0.4.2
jupyter-contrib-nbextensions==0.7.0
jupyter-highlight-selected-word==0.2.0
jupyter-matlab-proxy==0.5.1
jupyter-nbextensions-configurator==0.6.1
jupyter-remote-desktop-proxy==1.0.0
jupyter-server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1664327557912/work
jupyter-server-proxy==3.2.2
jupyter-telemetry @ file:///home/conda/feedstock_root/build_artifacts/jupyter_telemetry_1605173804246/work
jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1661522530937/work
jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1658332342865/work
jupyterhub @ file:///home/conda/feedstock_root/build_artifacts/jupyterhub-feedstock_1662986664893/work
jupyterlab @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_1664896434903/work
jupyterlab-pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1649936611996/work
jupyterlab-widgets==3.0.5
jupyterlab_server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server_1664216665765/work
jupytext==1.14.5
jwcrypto @ file:///home/conda/feedstock_root/build_artifacts/jwcrypto_1663340551184/work
kerchunk==0.1.0
keyring @ file:///home/conda/feedstock_root/build_artifacts/keyring_1671728228693/work
keyrings.alt @ file:///home/conda/feedstock_root/build_artifacts/keyrings.alt_1678128990932/work
kiwisolver==1.4.4
lazy_loader==0.1
libmambapy @ file:///home/conda/feedstock_root/build_artifacts/mamba-split_1678097005952/work/libmambapy
locket==1.0.0
looseversion @ file:///home/conda/feedstock_root/build_artifacts/looseversion_1677074853636/work
lxml==4.9.2
Mako @ file:///home/conda/feedstock_root/build_artifacts/mako_1663885412539/work
mamba @ file:///home/conda/feedstock_root/build_artifacts/mamba-split_1678097005952/work/mamba
Markdown==3.4.1
markdown-it-py==2.2.0
MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1648737547956/work
matlab-proxy==0.5.8
matlabengineforpython==9.13
matplotlib==3.7.1
matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
mdit-py-plugins==0.3.5
mdurl==0.1.2
mistune @ file:///home/conda/feedstock_root/build_artifacts/mistune_1657892024508/work
more-itertools @ file:///home/conda/feedstock_root/build_artifacts/more-itertools_1677514956219/work
mpi4py @ file:///home/conda/feedstock_root/build_artifacts/mpi4py_1667459939419/work
msgpack @ file:///home/conda/feedstock_root/build_artifacts/msgpack-python_1678312712169/work
multidict==6.0.4
multiprocess==0.70.14
mutagen @ file:///home/conda/feedstock_root/build_artifacts/mutagen_1665355730162/work
natsort==8.3.1
nbclassic @ file:///home/conda/feedstock_root/build_artifacts/nbclassic_1665246954203/work
nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1665125402713/work
nbconvert @ file:///home/conda/feedstock_root/build_artifacts/nbconvert-meta_1665098304483/work
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1664204096384/work
nbgitpuller==1.1.1
ndx-grayscalevolume==0.0.2
ndx-icephys-meta==0.1.0
ndx-spectrum==0.2.2
neo==0.12.0
nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1664684991461/work
netCDF4==1.6.3
networkx==3.0
neuroglancer==2.36
nibabel==5.0.1
nilearn==0.10.0
notebook @ file:///home/conda/feedstock_root/build_artifacts/notebook_1654636967533/work
notebook-shim @ file:///home/conda/feedstock_root/build_artifacts/notebook-shim_1646330736330/work
num2words==0.5.12
numcodecs @ file:///home/conda/feedstock_root/build_artifacts/numcodecs_1671214245674/work
numpy==1.23.5
nwbinspector==0.4.28
nwbwidgets==0.10.2
oauth2client==4.1.3
oauthlib @ file:///home/conda/feedstock_root/build_artifacts/oauthlib_1662766753257/work
ome-types==0.3.3
ome-zarr==0.6.1
orjson==3.8.7
osfclient==0.0.5
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1637239678211/work
pamela==1.0.0
pandas==1.5.3
pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work
parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
partd==1.3.0
pathos==0.3.0
patool @ file:///home/conda/feedstock_root/build_artifacts/patool_1667074693438/work
pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1602535608087/work
pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
Pillow @ file:///home/conda/feedstock_root/build_artifacts/pillow_1678273434437/work
Pint==0.20.1
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1633981968097/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1678652433955/work
plotly==5.13.1
pluggy==1.0.0
pooch==1.7.0
posix-ipc==1.1.1
pox==0.3.2
ppft==1.7.6.6
probeinterface==0.2.16
prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1649447152425/work
prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1662384672173/work
protobuf==4.22.1
psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1662356158604/work
ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
py2vega==0.6.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybids==0.15.6
pybind11==2.10.4
pycosat @ file:///home/conda/feedstock_root/build_artifacts/pycosat_1649384799612/work
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
pycryptodomex==3.17
pycurl==7.45.1
pydantic==1.10.6
pydra==0.22
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1660666458521/work
PyJWT @ file:///home/conda/feedstock_root/build_artifacts/pyjwt_1663432570896/work
pynwb==2.3.1
pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1663846997386/work
pyout==0.7.2
pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1652235407899/work
pyperclip @ file:///home/conda/feedstock_root/build_artifacts/pyperclip_1622337600177/work
pyrsistent @ file:///home/conda/feedstock_root/build_artifacts/pyrsistent_1649013351988/work
pysimdjson==5.0.2
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
pyspng-seunglab==1.0.0
pytest==7.2.2
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
python-gitlab @ file:///home/conda/feedstock_root/build_artifacts/python-gitlab_1675141094610/work
python-json-logger @ file:///home/conda/feedstock_root/build_artifacts/python-json-logger_1602545356084/work
python-jsonschema-objects==0.4.1
pythreejs==2.4.2
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1664798238822/work
PyWavelets==1.4.1
PyYAML==6.0
pyzmq @ file:///home/conda/feedstock_root/build_artifacts/pyzmq_1663830499178/work
quantities==0.14.1
rasterio==1.3.6
redis @ file:///home/conda/feedstock_root/build_artifacts/redis-py_1679315476006/work
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1661872987712/work
requests-ftp @ file:///home/conda/feedstock_root/build_artifacts/requests-ftp_1602638041394/work
requests-toolbelt @ file:///home/conda/feedstock_root/build_artifacts/requests-toolbelt_1667050328587/work
resource-backed-dask-array==0.1.0
rfc3339-validator==0.1.4
rfc3987==1.3.8
rsa==4.9
ruamel-yaml-conda @ file:///home/conda/feedstock_root/build_artifacts/ruamel_yaml_1653464396679/work
ruamel.yaml @ file:///home/conda/feedstock_root/build_artifacts/ruamel.yaml_1649033201098/work
ruamel.yaml.clib @ file:///home/conda/feedstock_root/build_artifacts/ruamel.yaml.clib_1649013063403/work
s3fs==2023.3.0
s3transfer==0.6.0
scikit-image==0.20.0
scikit-learn==1.2.2
scipy==1.10.1
SecretStorage @ file:///home/conda/feedstock_root/build_artifacts/secretstorage_1666848678066/work
semantic-version==2.10.0
Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1628511208346/work
simpervisor==0.4
simplejpeg==1.6.3
simplejson @ file:///home/conda/feedstock_root/build_artifacts/simplejson_1678881762010/work
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1662051266223/work
snuggs==1.4.7
soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work
spikeinterface==0.97.1
SQLAlchemy==1.3.24
stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1664126450622/work
tenacity==8.2.2
tensorstore==0.1.33
terminado @ file:///home/conda/feedstock_root/build_artifacts/terminado_1664482163879/work
threadpoolctl==3.1.0
tifffile==2023.3.15
tinycss2 @ file:///home/conda/feedstock_root/build_artifacts/tinycss2_1637612658783/work
toml==0.10.2
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1644342247877/work
toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1657485559105/work
tornado @ file:///home/conda/feedstock_root/build_artifacts/tornado_1656937830246/work
tqdm @ file:///home/conda/feedstock_root/build_artifacts/tqdm_1662214488106/work
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1663005918942/work
traittypes==0.2.1
trimesh==3.20.2
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1665144421445/work
ujson==5.7.0
uri-template==1.2.0
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1658789158161/work
wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1600965781394/work
webcolors==1.12
webencodings==0.5.1
webio-jupyter-extension==0.1.0
websocket-client @ file:///home/conda/feedstock_root/build_artifacts/websocket-client_1662334000707/work
websockify @ file:///home/conda/feedstock_root/build_artifacts/websockify_1674939817151/work
Whoosh @ file:///home/conda/feedstock_root/build_artifacts/whoosh_1667035213632/work
widgetsnbextension==4.0.5
wrapt @ file:///home/conda/feedstock_root/build_artifacts/wrapt_1677485519705/work
xarray==2023.2.0
xmlschema==2.2.2
xyzservices==2023.2.0
yarl==1.8.2
zarr @ file:///home/conda/feedstock_root/build_artifacts/zarr_1678980110903/work
zarr-checksum==0.2.8
zfpc==0.1.2
zfpy==1.0.0
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1665306627898/work
zope.event==4.6
zope.interface==6.0
zstandard==0.20.0

Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:36:39) [GCC 10.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib3
>>> urllib3.__version__
'1.26.11'

[jwodder]

@bendichter What are the cryptography and pyopenssl versions? The pip freeze output doesn't show them.

[jwodder]

@bendichter In addition, can you determine what installed packages depend on cryptography and pyopenssl? pip show cryptography pyopenssl might do it, but I'm not sure how that interacts with conda.

[bendichter]

@jwodder

$ pip show cryptography pyopenssl
Name: cryptography
Version: 39.0.2
Summary: cryptography is a package which provides cryptographic recipes and primitives to Python developers.
Home-page: https://github.com/pyca/cryptography
Author: The Python Cryptographic Authority and individual contributors
Author-email: [email protected]
License: (Apache-2.0 OR BSD-3-Clause) AND PSF-2.0
Location: /opt/conda/lib/python3.10/site-packages
Requires: cffi
Required-by: jwcrypto, pyOpenSSL, SecretStorage
---
Name: pyOpenSSL
Version: 22.0.0
Summary: Python wrapper module around the OpenSSL library
Home-page: https://pyopenssl.org/
Author: The pyOpenSSL developers
Author-email: [email protected]
License: Apache License, Version 2.0
Location: /opt/conda/lib/python3.10/site-packages
Requires: cryptography
Required-by: certipy

[jwodder]

@bendichter Please install pipdeptree, run it, and provide the output.

[bendichter]

@jwodder ok, I'd be happy to. Going forward, it might be easier for you to do this yourself. All you'd need to do is go to hub.dandiarchive.org and click on the last server option, then start a terminal.

[bendichter]

pipdeptree_output.txt

[jwodder]

@bendichter This is caused by a combination of (a) pyopenssl being present (due to being an indirect dependency of jupyterhub), (b) pyopenssl being out of date (thus leading to a mismatch with cryptography's version), and (c) botocore (a dependency of zarr_checksum) trying to import something from urllib3 that causes pyopenssl to be loaded. Upgrading pyopenssl (probably alongside cryptography as well) should fix this.

[bendichter]

@jwodder yup, that worked!

$ pip install -U pyopenssl
Requirement already satisfied: pyopenssl in /opt/conda/lib/python3.10/site-packages (22.0.0)
Collecting pyopenssl
  Downloading pyOpenSSL-23.1.1-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.9/57.9 kB 1.1 MB/s eta 0:00:00
Requirement already satisfied: cryptography<41,>=38.0.0 in /opt/conda/lib/python3.10/site-packages (from pyopenssl) (39.0.2)
Requirement already satisfied: cffi>=1.12 in /opt/conda/lib/python3.10/site-packages (from cryptography<41,>=38.0.0->pyopenssl) (1.15.1)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.10/site-packages (from cffi>=1.12->cryptography<41,>=38.0.0->pyopenssl) (2.21)
Installing collected packages: pyopenssl
  Attempting uninstall: pyopenssl
    Found existing installation: pyOpenSSL 22.0.0
    Uninstalling pyOpenSSL-22.0.0:
      Successfully uninstalled pyOpenSSL-22.0.0
Successfully installed pyopenssl-23.1.1
jovyan@jupyter-bendichter:~$ dandi download DANDI:000067/0.210812.1457
PATH                                     SIZE     DONE         DONE% CHECKSUM STATUS          MESSAGE    
PATH                                  SIZE     DONE            DONE% CHECKSUM STATUS          MESSAGE    
000067/dandiset.yaml                                                          skipped         no change  
...b-EE/sub-EE_ses-EE-044_ecephys.nwb 4.5 MB   4.5 MB           100%    ok    done                       ...b-EE/sub-EE_ses-EE-043_ecephys.nwb 5.5 MB   5.5 MB           100%    ok    done                       
...b-EE/sub-EE_ses-EE-081_ecephys.nwb 35.1 MB  16.8 MB           47%          downloading                
...b-EE/sub-EE_ses-EE-080_ecephys.nwb 172.8 MB 8.4 MB             4%          downloading                
...b-EE/sub-EE_ses-EE-080_ecephys.nwb 172.8 MB 16.8 MB            9%          downloading                  .b-EE/sub-EE_ses-EE-083_ecephys.nwb 287.3 MB 8.4 MB             2%          downloading                
...b-EE/sub-EE_ses-EE-083_ecephys.nwb 287.3 MB 16.8 MB            5%          downloading                
...b-EE/sub-EE_ses-EE-042_ecephys.nwb 191.0 MB            

Now the question is how to prevent this in the future? Can we be more strict about the dependencies for dandi cli? Maybe adding pyopenssl >= 23.1.1 would work, or maybe we could just put a version constraint on zarr_checksum?

[jwodder]

@bendichter

Maybe adding pyopenssl >= 23.1.1 would work,

pyopenssl is not a dependency (direct or indirect) of dandi-cli. This problem can only occur when other packages are installed alongside dandi-cli.

maybe we could just put a version constraint on zarr_checksum?

I'm unclear what constraint you're imagining.

[bendichter]

I'm just trying to come up with a modification to the dandi-cli library that would fix this for new installations of dandi-cli to prevent this type of installation error in the future, but I'm not exactly sure that that would be. We either need to do that or we should at least create a fix for the MATLAB environment of DANDI Hub.

[satra]

@bendichter - this may be worth pinging the mathworks/metacell folks about. the container is derived directly from a container built by mathworks. unless this has been already addressed by mathworks, feel free to send a PR to add the extra upgrade for now to the matlab dockerfile in this repo. they don't version their containers, so it's hard to know if they have already changed the base.

[bendichter]

@satra I'd like to help but I am a bit in the dark here. How would I check if this has been fixed, who would I contact, and what would I ask them to change?

[CodyCBakerPhD]

If pip install -U pyopenssl is all it takes to fix, shouldn't it be possible to fix this directly on the Hub by adding an extra RUN command at this point in the Dockerfile and re-build the Hub?

That's at least a spot fix, though how to systematically test that it holds for future DANDI CLI releases is beyond me

[bendichter]

@satra , do you think that would work?

[satra]

sure. works for me. one could add a cli-test action here in the repo to make sure it works, but we tend to build the image directly on dockerhub.