AttributeError: module 'pyarrow.lib' has no attribute 'ListViewType' about datasets

[albertvillanova]

Please note that the error is raised just at import:

import pyarrow.parquet as pq

Therefore it must be caused by some problem with your pyarrow installation. I would recommend you uninstall and install pyarrow again.

I also see that it seems you use conda to install pyarrow. Please note that pyarrow offers 3 different packages in conda-forge: https://arrow.apache.org/docs/python/install.html#using-conda

conda install -c conda-forge pyarrow

While the pyarrow conda-forge package is the right choice for most users, both a minimal and maximal variant of the package exist, either of which may be better for your use case. See Differences between conda-forge packages.

Please, make sure you install the right one: I guess it is either pyarrow (or pyarrow-all).

[NicoNicoNico123]

I have same issue, please downgrade pyarrow==15.0.2, it seem datasets library need to be fix

[albertvillanova]

It is not a problem with the datasets library: we support latest version of pyarrow and our Continuous Integration tests are using pyarrow 16.1.0 without any problem.

The error reported here is raised when importing pyarrow.parquet:

---> 29 import pyarrow.parquet as pq

File /opt/conda/lib/python3.10/site-packages/pyarrow/parquet/__init__.py:20
      1 # Licensed to the Apache Software Foundation (ASF) under one
      2 # or more contributor license agreements.  See the NOTICE file
      3 # distributed with this work for additional information
   (...)
     17 
     18 # flake8: noqa
---> 20 from .core import *

File /opt/conda/lib/python3.10/site-packages/pyarrow/parquet/core.py:33
     30 import pyarrow as pa
     32 try:
---> 33     import pyarrow._parquet as _parquet
     34 except ImportError as exc:
     35     raise ImportError(
     36         "The pyarrow installation is not built with support "
     37         f"for the Parquet file format ({str(exc)})"
     38     ) from None

File /opt/conda/lib/python3.10/site-packages/pyarrow/_parquet.pyx:1, in init pyarrow._parquet()

AttributeError: module 'pyarrow.lib' has no attribute 'ListViewType'

This can only be explained if pyarrow was not properly installed.

If the user just installed pyarrow-core from conda-forge, then its parquet subpackage is not installed and cannot be imported. You can check pyarrow docs:

• Differences between conda-forge packages: https://arrow.apache.org/docs/python/install.html#python-conda-differences

The pyarrow-core package includes the following functionality:
...
The pyarrow package adds the following:
...
Parquet (i.e., pyarrow.parquet)