phyks / bmc Goto Github PK

View Code? Open in Web Editor NEW

16.0 16.0 4.0 3.23 MB

BMC (BiblioManagementClient) is a simple script to download and store your articles.

License: Other

Python 98.22% TeX 1.78%

bmc's People

Contributors

Stargazers

Watchers

Forkers

drvinceknight m000 bcbnz armavica

bmc's Issues

Discussion: what should be the `delete` behavior?

For a file in Papers
- Remove the citation from index.bib and unlink the file (currently used, most logical);
- Remove the citation, do not unlink the file (not logical at all).
For a file imported in-place:
- Remove from index, unlink (current behavior, somewhat objectionable)
- Remove from index, leave in-place (not implemented).

The user should be forewarned that some operations are a) irreversible and/or b) not idempotent, IMHO. An alternative - give the delete command an -m/manual switch to signify no unlinking is to take place. Another, perhaps more usable alternative, do not unlink by default.

Release

Hi,

Is it possible to have a release? I would like to prepare a package for archlinux. I think having identified versions make things easier to follow from the user point of view (in terms of evolution, regressions, bugs, features...)

Manual entry not working?

Hi I'm just getting started with BMC. I get an error when pointing it at an old pdf for which I can't seem to find the DOI so hoping to use manual:

~$ python BMC/bmc.py import ~/Desktop/yadin_1965.pdf
DOI / arXiv / ISBN / manual / skip? manual
The bibtex entry found for /Users/vince/Desktop/yadin_1965.pdf is:
Traceback (most recent call last):
  File "BMC/bmc.py", line 550, in <module>
    args.tag)
  File "BMC/bmc.py", line 188, in addFile
    bibtex = checkBibtex(src, bibtex_string)
  File "BMC/bmc.py", line 30, in checkBibtex
     bibtex = bibtex[list(bibtex.keys())[0]]
IndexError: list index out of range

Have I got the use case wrong for manual or is it not implemented yet?

Archlinux package

Hi,

FYI, I made a package for archlinux: https://aur.archlinux.org/packages/bmc

Move to python3

All the dependencies should be ok with python3, except the socks module. I'll have to look for something, but the rest should not be much work to be py3 compatible.

Ensure `Papers` dir exists

The script fails sometimes when the Papers dir does not exist. It should create it in such cases.

Discussion: allow non-ASCII alphanumeric characters in filename slugs

I know I'm opening a can'o'worms with this one (at least on Python 2) :(, yet sometimes there's a need to add a non-ASCII book/article to the library (Latin-1, Japanese, Cyrillic, without transcription). I fiddled a bit with slugify in libbmc/tools.py, but did not devote enough time. I'm leaving this featreq in hope of later coming back with a pull.

Watermarks

Hi,

Just for your information, about the last bullet point of your goal list, you can have a look to this simple script: https://github.com/sciunto/tear-pages

PS: I think we have similar needs in bibliography management :)

bmc.py KeyError

'bmc.py download http://arxiv.org/pdf/1412.3638v2'

'''
Downloading http://arxiv.org/pdf/1412.3638v2
[==================================================] 96%Download finished
ArXiv id for /tmp/tmpbwpxkzwh.pdf is 1412.3638v2.
Traceback (most recent call last):
File "/usr/bin/bmc.py", line 568, in
args.tag)
File "/usr/bin/bmc.py", line 300, in downloadFile
new_name = addFile(tmp.name, filetype, manual, autoconfirm, tag)
File "/usr/bin/bmc.py", line 173, in addFile
bibtex = fetcher.arXiv2Bib(arxiv).strip().replace(',', ",\n")+"\n"
File "/usr/lib/python3.5/site-packages/libbmc/fetcher.py", line 323, in arXiv2Bib
return tools.parsed2Bibtex(fetched_bibtex)
File "/usr/lib/python3.5/site-packages/libbmc/tools.py", line 49, in parsed2Bibtex
bibtex = '@'+parsed['type']+'{'+parsed['id']+",\n"
KeyError: 'type'
'''

Sometimes author (or some fields) are not fetched with ISBN

This is due to xlcnd/isbnlib#3 (comment) and will be solved with the next version of this module.

Feature request: tag name completion from the set of existing directories

Rationale: simplification of workflow, otherwise one has to keep two terminals side by side to be sure of tag spelling.

virtuanenv - module not found

I'm trying to install BMC on my arch linux box inside of a python3 virtual environment.

$ python -m venv biblio
$ source biblio/bin/activate
(biblio)$ pip install arxiv2bib PySocks bibtexparser pyPDF2 isbnlib
(biblio)$ git clone https://github.com/Phyks/BMC
(biblio)$ ./BMC/bmc.py

Traceback (most recent call last):
  File "./bmc.py", line 11, in <module>
    from bibtexparser.bparser import BibTexParser
ImportError: No module named bibtexparser.bparser

however, import bibtexparser from a python3 console inside the virtualenv seems to work.

Any idea whats going on?

Feature request: ability to work with preprints/technical reports

@TechReport{Doe2015,
  author =   {John A. Doe},
  title =    {How to frobnify efficiently: lessons from the 10th International Frobnification Experiment},
  institution =  {Scripps Institution of Oceanography},
  year =     {2015},
  type =     {Working Paper},
  number =   {135/16},
  address =      {La Jolla, Calif.},
  month =    {December},
}

If I manually add the BibTeX record now, BMC won't recognize the record type.

typeError after download with python3

[==================================================] 96%Download finished
Traceback (most recent call last):
  File "/home/berceanu/tmp/biblio/bin/bmc.py", line 532, in <module>
    args.tag)
  File "/home/berceanu/tmp/biblio/bin/bmc.py", line 292, in downloadFile
    fh.write(dl)
  File "/usr/lib/python3.4/codecs.py", line 711, in write
    return self.writer.write(data)
  File "/usr/lib/python3.4/codecs.py", line 368, in write
    data, consumed = self.encode(object, self.errors)
TypeError: Can't convert 'bytes' object to str implicitly

bmc.py list doesn't list files that have been imported in-place

84a7a1c : list command doesn't output paths to files outside the ~/Papers directory.

import from zotero

could you add import capability to BMC, allowing one to migrate a zotero library for instance?

Refactor config file

Hi,

I think using a python file for parameters is not the best choice.
A config file (json, yaml...) located in ~/.config would be much better.

Use isbnlib

Cf xlcnd/isbntools#55

Missing files in test/

In your directory test, you call many missing files. See below :)


======================================================================

ERROR: test_arXiv2Bib (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_arXiv2Bib_False (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_doi2Bib (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_doi2Bib_False (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_download (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_download_invalid_type (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_download_invalid_url (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findArXivId (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findDOI_DJVU (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findDOI_False (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findDOI_PDF (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findHALId (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findISBN_DJVU (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findISBN_False (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_findISBN_PDF (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_isbn2Bib (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

======================================================================

ERROR: test_isbn2Bib_False (test_fetcher.TestFetcher)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/build/sciunto/BMC/tests/test_fetcher.py", line 18, in setUp

with open("tests/src/doi.bib", 'r') as fh:

IOError: [Errno 2] No such file or directory: 'tests/src/doi.bib'

incorrect bibinfo extraction for certain arxiv papers

The following paper
http://arxiv.org/abs/1107.4487
is incorrectly reported as being
http://dx.doi.org/10.1038/nphys1959
I have more examples, if needed for writing tests.

Refactor

I am again using this software heavily recently, and have been working on a better associated library, which is now available from here.

Hence, there will be a refactor to use this library rather than duplicating it in the source code.

I also plan on making BMC behave a bit as Git, with a repo and to work in this repo, allowing more easily to keep papers in different places.

Feel free to give feedbacks about issues with the current design, arguments against new design etc here.

Im getting some error.

I have installed BMC but cannot get it working.
This is the usual error:

bmc.py import vlada.pdf
DOI for vlada.pdf is 10.1002/wrna.1254.
Traceback (most recent call last):
File "/usr/local/bin/bmc.py", line 584, in
args.tag, not args.inplace)
File "/usr/local/bin/bmc.py", line 185, in addFile
bibtex_string = tools.parsed2Bibtex(bibtex)
File "/usr/local/lib/python2.7/dist-packages/libbmc/tools.py", line 49, in parsed2Bibtex
bibtex = '@'+parsed['type']+'{'+parsed['id']+",\n"
KeyError: u'type'

I have bibtexparser 0.6.1 installed. Can it be the problem?
Thanks.
Petro

Leaving the imported file in place

Is it possible to add an option to leave the file where it currently is?

failed to download pdf from rev. mod. phys

Error while downloading the following paper
http://journals.aps.org/rmp/pdf/10.1103/RevModPhys.85.299

Traceback (most recent call last):
  File "/home/berceanu/tmp/biblio/bin/bmc.py", line 532, in <module>
    args.tag)
  File "/home/berceanu/tmp/biblio/bin/bmc.py", line 285, in downloadFile
    dl, contenttype = fetcher.download(url)
  File "/home/berceanu/tmp/biblio/lib/python3.4/site-packages/libbmc/fetcher.py", line 71, in download
    r = urlopen(url)
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 473, in _open
    '_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1202, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.4/urllib/request.py", line 1177, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.4/http/client.py", line 1172, in getresponse
    response.begin()
  File "/usr/lib/python3.4/http/client.py", line 351, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.4/socket.py", line 371, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

Unable to open index file

from a fresh today install I get

$ bmc.py list 
WARNING:  Unable to open index file.
$

idem for resync::

$ bmc.py resync
Resync files and bibtex index? [y/N] y
WARNING:  Unable to open index file.
$

New version?

Hi,

Do you plan to release a new version? 0.2 does not include the setup.py and many other improvements. :)

Thanks!