nicolasshu / zotero2papis

Why

When I wrote this, I had found out about the repository papis which is pretty nice. However, when I checked out the papis-zotero repository, I realized that the repository was a bit outdated, some of its documentation was a little outdated, and it didn’t particularly do what I wanted it to do. To my understanding, the package described in papis-zotero is setup to do a single copy of the files described in the zotero.sqlite. But the problem is that not all of the attachments are copied over, and the way the files are organized is that it saves the database keys as the names of the folders where the PDF files end up being copied to.

What I particularly wanted was a tool which allowed one to parse the information from Zotero to papis in a way that it would keep some sort of human-readable file structure, and keep the attachments used on Zotero. How I then decided to design this is to make this a one-way workflow which looks through the SQLite file from Zotero, and copies them over to the papis directory.

Installation

First clone the repository and go into the repository.

git clone https://github.com/nicolasshu/zotero2papis

To set it up, you may either use as an editable package:

pip install -e .

Or as a binary package on your environment:

python3 setup.py install
# or
pip install .

Usage

To use it, simply run

zotero2papis -z {location_of_zotero} -o {output_location}

where the -z / --zotdir will be the parent directory of where your Zotero database is located (e.g. ~/Zotero or ~/sync/zotero) and -o / --outdir will be the output directory. If the output directory does not exist, then the program will make the appropriate directories for you.

This will then start copying the files to the output directory. It will be first looking through the database to see where the files or linked files are originally located.

• If the output directory is the same directory that the PDF files are already stored, then:
  • It will copy the attachments from the Zotero storage to the output directory
  • It will create the YAML files in the output directory
• If the output directory is at a different location, then:
  • It will copy the PDFs from the location they are originally stored to the output directory
  • It will copy the attachments from the Zotero storage to the output directory
  • It will create the YAML files in the output directory

At this point, you can start to use papis on the output directory or to copy the contents to your desired papis library. In case you do not see any updates on your papis despite you having changed the directory or moved the files, clear the cache by running the following as suggested in the FAQ.

papis --clear-cache

zotero2papis's Issues

defaultFile is not defined

Clean directory ~/Dokumenty/clanky/bibliography and then:

fun~/D/clanky$ zotero2papis -z ~/.local/share/zotero/ -o ~/Dokumenty/clanky/bibliography/ --
verbose
===================================
  TITLE: Declaration "Dominus Iesus" On the Unicity and Salvific Universality of Jesus Christ and the Church
AUTHORS: Ratzinger, Joseph
===================================
    Now parsing other storage files...
-----------------------
    KEY   DNTRT475
    PATH  storage:rc_con_cfaith_doc_20000806_dominus-iesus_en.html
    PARID 2
      Storage File: rc_con_cfaith_doc_20000806_dominus-iesus_en.html
      :Orig:  /home/matej/.local/share/zotero/storage/DNTRT475/rc_con_cfaith_doc_20000806_dominus-iesus_en.html
      :Dest:  /home/matej/Dokumenty/clanky/bibliography/2000_Declaration "Dominus Iesus" On the Unicity and Salvific Universality of Jesus Christ and the Church/rc_con_cfaith_doc_20000806_dominus-iesus_en.html
       - Copied



===================================
  TITLE: Tomáš Halík: smířená různost: rozhovor
AUTHORS: Halík, Tomáš
===================================
    Now parsing other storage files...
Traceback (most recent call last):
  File "/home/matej/.bin/zotero2papis", line 33, in <module>
    sys.exit(load_entry_point('zotero2papis', 'console_scripts', 'zotero2papis')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matej/archiv/knihovna/repos/tmp/zotero2papis/zotero2papis/zotero2papis.py", line 549, in run
    client.run()
  File "/home/matej/archiv/knihovna/repos/tmp/zotero2papis/zotero2papis/zotero2papis.py", line 513, in run
    files, target_dir, deleted_entry = self.getFiles(conn, itemID, itemKey)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matej/archiv/knihovna/repos/tmp/zotero2papis/zotero2papis/zotero2papis.py", line 421, in getFiles
    if files == [] and defaultFile:
                       ^^^^^^^^^^^
NameError: name 'defaultFile' is not defined
fun~/D/clanky$ 

Using papis 0.12 and Zotero 6.0.26 from Flatpak.

zotero.sqlite.gz

object is not subscriptable

I am getting the following error:

Traceback (most recent call last):
  File "/home/jt/.local/bin/zotero2papis", line 8, in <module>
    sys.exit(run())
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/jt/.local/lib/python3.10/site-packages/zotero2papis/zotero2papis.py", line 549, in run
    client.run()
  File "/home/jt/.local/lib/python3.10/site-packages/zotero2papis/zotero2papis.py", line 513, in run
    files, target_dir, deleted_entry = self.getFiles(conn, itemID, itemKey)
  File "/home/jt/.local/lib/python3.10/site-packages/zotero2papis/zotero2papis.py", line 380, in getFiles
    if path[:8] != "storage:": continue
TypeError: 'NoneType' object is not subscriptable

I do not have much experience with python so I did not look into the code. Running up to date arch. Any ideas what could be causing this?

FileNotFoundError: [Errno 2] No such file or directory: 'storage:XXXXXXX.pdf'

Hi. The script doesn't work for me - the "path" variable seems to not be defined properly, see output below. The file in question exists at /home/XXX/.zotero_data/storage/ADENEUW7/Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf

I am wondering if I am doing something wrong? Not only does "path" contain a "storeage:" stub that should not be there, it also doesn't contain the full path but only the file name.

> zotero2papis --zotdir /home/XXX/.zotero_data -o /home/XXX/Papers --verbose

 (1, 'journalArticle', 'IYZZDBTY', '2019-08-08 09:37:23', '2022-04-20 12:51:29', '2022-04-20 12:51:29')
Heterogeneity and Aggregation
    KEY   ADENEUW7
    PATH  storage:Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf
    PARID 1
    DIR:  
      original: /home/XXX/.zotero_data/storage/ADENEUW7/storage:Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf
      dest:     /home/XXX/Papers/storage:Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf
storage:Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf

Traceback (most recent call last):
[...]
FileNotFoundError: [Errno 2] No such file or directory: 'storage:Blundell and Stoker - 2005 - Heterogeneity and Aggregation.pdf'