load_krebsregister() fails with [SSL: CERTIFICATE_VERIFY_FAILED] about recordlinkage

[J535D165]

Thanks for reporting this issue. Quite some info on the internet about this problem (https://access.redhat.com/articles/2039753).

I did change https:// to http:// in commit 30312c0. Does this solve your issue?

[twilight2001]

I also have problem with this line, but I get a "ValueError: Unrecognised compression type: zip" error instead ... the error is different from the above: (this is on Win 7 Pro, 64 bit, running Anaconda)

krebs_data, krebs_match = load_krebsregister(missing_values=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Python\Python27\site-packages\recordlinkag
e\datasets\external.py", line 75, in load_krebsregister
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "C:\Users\user\AppData\Roaming\Python\Python27\site-packages\recordlinkag
e\datasets\external.py", line 123, in _krebsregister_block
    compression='zip')
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 529, in parse
r_f
    return _read(filepath_or_buffer, kwds)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 295, in _read

    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 612, in __ini
t__
    self._make_engine(self.engine)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 747, in _make
_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "C:\Anaconda2\lib\site-packages\pandas\io\parsers.py", line 1119, in __in
it__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas\parser.pyx", line 353, in pandas.parser.TextReader.__cinit__ (pan
das\parser.c:3246)
  File "pandas\parser.pyx", line 567, in pandas.parser.TextReader._setup_parser_
source (pandas\parser.c:5787)
ValueError: Unrecognized compression type: zip

[J535D165]

Can you show the output of

import pandas
pandas.show_versions()

It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")?

[twilight2001]

Hi Jon, Thanks for your follow up again, we tested it on Linux and everything works down to ECM algorithm. My goal was to get some estimates of M and U with frequency dependence to verify our scoring (we have one-to-one matching data, about 1 million records of reference address)... right now our computed numbers of m and u looks crazy... The following is pandas version info from Windows/Anaconda 2 latest version: pandas: 0.18.0 nose: 1.3.7 pip: 9.0.1 setuptools: 20.3 Cython: 0.23.4 numpy: 1.10.4 scipy: 1.0.0 statsmodels: 0.6. xarray: None IPython: 4.1.2 sphinx: 1.3.5 patsy: 0.4.0 dateutil: 2.5.1 pytz: 2016.2 blosc: None bottleneck: 1.0.0 tables: 3.2.2 numexpr: 2.5 matplotlib: 1.5.1 openpyxl: 2.5.0a2 xlrd: 0.9.4 xlwt: 1.0.0 xlsxwriter: 0.8.4 lxml: 3.6.0 bs4: 4.4.1 html5lib: None httplib2: None apiclient: None sqlalchemy: 1.0.1 pymysql: None psycopg2: None jinja2: 2.8 boto: 2.39.0 best regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

[twilight2001]

Yes, something is wrong with my winzip (seems overridden by 7z or winrar), need reboot to test again ...

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

[twilight2001]

Hi Jonathan, After reinstalling winzip v12, pandas can read csv files in my working directory now. But rerunning krebs_data, krebs_match = load_krebsregister(missing_values=0) I still get ValueError: Unrecognized compression type: zip regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

[twilight2001]

Hi Jonathan, Wonder if you consider it feasible to build this hybrid model on top of the toolkit: known as the cocktail approach, it combines probabilistic (EM) and rule based approaches in a flow. First, use EM to figure out what variables are most relevant, then build rules on top of such variables (see figure from Shah) : http://digitalcommons.calpoly.edu/theses/1353/ http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.218.9234&rep=rep1&type=pdf Shah also added relational linkage information to improve accuracy which is very interesting... best regards Geoff

…

On 6 November 2017 at 17:21, Jonathan de Bruin ***@***.***> wrote: Can you show the output of import pandas pandas.show_versions() It looks like an issue with your installation of pandas. Can you read a zipped csv file with pandas.read_csv('yourfile_csv.zip")? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALDDbhMAMk6mkzYTlJzIEbsreaAO2epSks5szs9_gaJpZM4POFjF> .

[Michael-E-Rose]

Problem prevails with recordlinkage version 0.11.2

>>> from recordlinkage.datasets import load_krebsregister
>>> krebs_data, krebs_match = load_krebsregister(missing_values=0)
Start downloading the data.
Issue with downloading the data: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_10.zip'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 75, in load_krebsregister
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 75, in <listcomp>
    data = pandas.concat([_krebsregister_block(bl) for bl in block])
  File "/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/external.py", line 123, in _krebsregister_block
    compression='zip')
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 709, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 449, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 818, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1049, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python3.5/dist-packages/pandas/io/parsers.py", line 1695, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 402, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 664, in pandas._libs.parsers.TextReader._setup_parser_source
  File "/usr/lib/python3.5/zipfile.py", line 1009, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_1.zip'

[J535D165]

Hello, this is a different issue. You do not have permission to write /usr/local/lib/python3.5/dist-packages/recordlinkage/datasets/krebsregister/block_10.zip.

Is it possible to get write access?

[Michael-E-Rose]

Yes it is, I invoked python3 with sudo. I wasn't aware of what the problem was in this case.

Thanks for the help!

[Dragut]

I encountered the same problem when doing the load:
krebs_X, krebs_true_links = load_krebsregister(missing_values=0)
Error Message:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\s59018\AppData\Roaming\Python\Python36\site-packages\recordlinkage\datasets\krebsregister\block_1.zip'
Please help.

[J535D165]

This is a permission error. Do you have permission to the given folder?

I will work on a permanent solution.

[Dragut]

I rerun it successfully from home rather than in the office. Looks like a connection problem which fails to download the data completely.