dougy147 / scitopdf

bash script to quickly fetch, download and open scientific papers from Sci-Hub's database.

scitopdf

In general terms, scitopdf is a minimalist program for avid readers of scientific papers (that... I'm not).

More specifically, but not enough, scitopdf quickly leverages Crossref search engine and Sci-Hub's database.

Feed it with some infos about the paper, and 5 seconds later (maybe less... or more) it will pop in front of your eyes 👀.

scitopdf recognizes URL and DOI when given, else it will process any info about the paper to find it : (trunkated) title || authors || journal || DOI || URL || year of publication || ...

Install

If you really care about your disk space, don't install ! And try it like this :

paper="the paper you are looking for"
curl https://raw.githubusercontent.com/dougy147/scitopdf/master/bin/scitopdf | bash -s ${paper}

If 12KB is no problem to you, push your floppy diskette in, and follow those lines :

git clone https://github.com/dougy147/scitopdf
cd scitopdf
sudo make install

Also check your distro repos 👀

• Arch Linux (AUR) : yay -S scitopdf-git

• ROSA Linux (official repository) : sudo dnf install scitopdf

Adapt to your needs !

Desired features for scitopdf are {simplici-compatibili-flexibili}docious.

-- Mary Poppins

That goal is obviously not reached (too many OS 🤯, sad country restrictions 😥, coding skills deficit 📉...) but let's start somewhere!

Flag	Functionality
-l, --list	Download references line by line from a bibliography file
-D, --download-dir	Store papers in a specified directory (absolute path)
-u, --url	Manually set Sci-Hub address
-p, --no-auto-open	Pass on auto-opening
-q, --quiet	Quiet mode, no echo except for errors
-h, --help	Print this help menu
man scitopdf	Check the manual for more tweaks

• Example : scitopdf -p "protein measurement with the folin" -q -D "$HOME/science"

Some hints

Papers are stored

Papers are automatically saved to $XDG_DOWNLOAD_DIR/scitopdf (or ~/Downloads/scitopdf depending on your system). You can change that default directory (pre-install) from the script itself (./bin/scitopdf).

Be sure you can read

For auto-opening, scitopdf will do whatever it can to use your default PDF reader. But if it fails, don't forget to set a READER environment variable, or to install zathura (a minimalist and lightweight document viewer).

"I want it all"

If you're very hungry for papers, you can auto-download multiple ones. Just give scitopdf a list of references (contained in a .txt file, or whatever, as long as references are listed line by line).

So, paste a bibliography in bib.txt, and scitopdf -l bib.txt will process each line and download every paper it finds (no auto-opening).

Don't waste time typing!

Go even faster and make your own keyboard shortcut. Map your favorite keys combination (I use Super+S) to scitopdf $(xclip -o) (or any other clipboard tool). Now, just highlight any title, DOI or URL you come accross, press your keys, and you're all set for a read.

That's full speed accessing your papers now. Close enough from speed of thought 👍. (why am I selling that dumb script so much?)

"But I love typing..."

If you're a typewriting kind of person and prefer it the old way, well, no judgment. scitopdf reveals itself pretty cool too in the terminal, or when combined with dmenu.

Map this search=$(printf "" | dmenu -p "Scitopdf") && scitopdf "$search" to your favorite keys combo.

macOS compatibility

macOS runs a dinosaur version of bash. But scitopdf won't allow its users to be left behind. That's why its code is meant to stay retro-compatible. How kind from this little script 😎!

Instructions for the careless mind

Power is all relative, and this program's is pretty low; even quite harmless. However, I must release myself of any responsibility in the way you will use this program. Indeed, its use might be unappropriate in your country.

As far as I understand (i.e. poorly), imperative programming is like stacking bricks on top of each other. If there's nothing wrong stacking, climbing the pile might. That way, it's unnecessary to say I'm only responsible for sharing a recipe one could find all the ingredients thanks to any search engine.

So, illegality is not contained in that program. It can only be in its user's behavior. Please use that script consciously, with and on your personal goods only.

Special thanks to contributors

Feel free 😎

If scitopdf has been of any help to you, I'd be glad and thankful !

BTC : bc1q4cflj0e3hwcn5edut654je86upn37p37gut5yk

scitopdf's Issues

Saves html code of site not pdf

Each time I try to run this, it downloads the html code for a sci-hub page (I believe something the file not found page), in place of the pdf. An example is shown below;

<html>
    <head>
	 <title>Sci-Hub - search proxy to download article</title>
	<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
        <meta name="viewport" content="width=device-width, heihht=device-height, initial-scale=1.0">
	<meta name="keywords" content="sci-hub,scihub">
	<meta name="description" content="The first pirate website in the world to open mass and public access to tens of millions research papers">
	<meta property="og:image" content="//img.sci-hub.shop/scihub/logo_en.png"/>
        <link rel="alternate" hreflang="ru" href="//sci-hub.tf/lang/ru" />
        <link rel="alternate" hreflang="en" href="//sci-hub.tf/lang/en" />
	<style type = "text/css">
		html, body, div, p, ul { margin: 0; padding: 0; font-family: Avenir }
                
                
                body { background: url('//img.sci-hub.shop/scihub/map.jpg') no-repeat; background-size: contain }
               
                #title { font-family: Tahoma; font-size: 250%; text-align: center; color: #993333 }
                #first { margin: 2%; font-size: 100%; text-align: center; color: #993333 }
                #desc p { margin: 3%;font-size: 100%; text-align: justify }
                #mission { margin-top: 4%; font-size: 120%; text-align: center; color: #993333 }
                
                #social { margin-top: 6%; text-align: center; color: #993333 }
                #social img { border: 0; margin: 2% }
				h1 {color:#993333;margin:0;margin-bottom:24px}
		a:hover {color:darkgreen}
                a {color:#aaa;margin:0;margin-bottom:24px}
                a#back { display: block; text-decoration: none; margin-top: 8%; width: 100%; padding: 2% 0 2% 0; background-color: #993333; color: white; text-align: center; font-size: 100% }
                img{max-width:100%}
				#message { text-align:left;color:#aaa;font-family: Verdana;font-size:16px;margin:32px; width: 480px }
                #noproxy { text-align:center;color:#aaa;font-family: Verdana;font-size:18px;margin-top:32px;display:none; }
                #found { text-align:center;color:green;font-family: Verdana;font-size:22px;margin-top:32px;display:none; }
	</style>
    </head>
<body>
    <div id ="about">
        <div id = "title"><h1><a href = "//sci-hub.wf">Sci-hub</a></h1></div>
        <div id = "first"><h1>
<p>Sorry, sci-hub has not included this article yet</p>
<p>You can register and log in to the <a href="http://www.wosonhj.com" target="_blank">Mutual Aid-Science Community,</a></p>
<p>and get it by posting for help</p>
<p><a href="http://www.wosonhj.com/suggest/22xs.html" target="_blank">Mutual Aid-Science Community Instructions</a></p>
<img src="https://img.sci-hub.shop/misc/img/maid1.png" height="306" width="918">
<br>
Please try to search again using DOI. DOI is the unique identifier of thesis, and searching through DOI can more accurately find the corresponding thesis documents.</h1>
		<h1><a href = "//sci-hub.wf/Find-DOI.html" target="_blank">How to quickly find the DOI number of an article</a></h1>
		<p>If you still cannot find thesis through DOI, we will include relevant articles as soon as possible,please try searching the corresponding DOI again after a while.</p>
		<div id = "message">&#9432; you can close this page and check later if the article has been downloaded</div>
                        <div id = "noproxy">no matching proxies found</div>
                        <div id = "found">proxy found, please wait</div>
<br></br>
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-7696326278603752"
     crossorigin="anonymous"></script>
<ins class="adsbygoogle"
     style="display:inline-block;width:970px;height:90px"
     data-ad-client="ca-pub-7696326278603752"
     data-ad-slot="4246281558"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
<p></div>
<script>var allurl=window.location.href;window.history.pushState({}, 0, "https://" + window.location.host);</script>
<script>setTimeout(function() {  window.history.pushState({}, 0, allurl);  }, 1000);</script>
    </div>
    <a id ="back" href = "/">&larr; return to main</a>
</body>
</html>


<html>
    <head>
	 <title>Sci-Hub - search proxy to download article</title>
	<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
        <meta name="viewport" content="width=device-width, heihht=device-height, initial-scale=1.0">
	<meta name="keywords" content="sci-hub,scihub">
	<meta name="description" content="The first pirate website in the world to open mass and public access to tens of millions research papers">
	<meta property="og:image" content="//img.sci-hub.shop/scihub/logo_en.png"/>
        <link rel="alternate" hreflang="ru" href="//sci-hub.tf/lang/ru" />
        <link rel="alternate" hreflang="en" href="//sci-hub.tf/lang/en" />
	<style type = "text/css">
		html, body, div, p, ul { margin: 0; padding: 0; font-family: Avenir }
                
                
                body { background: url('//img.sci-hub.shop/scihub/map.jpg') no-repeat; background-size: contain }
               
                #title { font-family: Tahoma; font-size: 250%; text-align: center; color: #993333 }
                #first { margin: 2%; font-size: 100%; text-align: center; color: #993333 }
                #desc p { margin: 3%;font-size: 100%; text-align: justify }
                #mission { margin-top: 4%; font-size: 120%; text-align: center; color: #993333 }
                
                #social { margin-top: 6%; text-align: center; color: #993333 }
                #social img { border: 0; margin: 2% }
				h1 {color:#993333;margin:0;margin-bottom:24px}
		a:hover {color:darkgreen}
                a {color:#aaa;margin:0;margin-bottom:24px}
                a#back { display: block; text-decoration: none; margin-top: 8%; width: 100%; padding: 2% 0 2% 0; background-color: #993333; color: white; text-align: center; font-size: 100% }
                img{max-width:100%}
				#message { text-align:left;color:#aaa;font-family: Verdana;font-size:16px;margin:32px; width: 480px }
                #noproxy { text-align:center;color:#aaa;font-family: Verdana;font-size:18px;margin-top:32px;display:none; }
                #found { text-align:center;color:green;font-family: Verdana;font-size:22px;margin-top:32px;display:none; }
	</style>
    </head>
<body>
    <div id ="about">
        <div id = "title"><h1><a href = "//sci-hub.wf">Sci-hub</a></h1></div>
        <div id = "first"><h1>
<p>Sorry, sci-hub has not included this article yet</p>
<p>You can register and log in to the <a href="http://www.wosonhj.com" target="_blank">Mutual Aid-Science Community,</a></p>
<p>and get it by posting for help</p>
<p><a href="http://www.wosonhj.com/suggest/22xs.html" target="_blank">Mutual Aid-Science Community Instructions</a></p>
<img src="https://img.sci-hub.shop/misc/img/maid1.png" height="306" width="918">
<br>
Please try to search again using DOI. DOI is the unique identifier of thesis, and searching through DOI can more accurately find the corresponding thesis documents.</h1>
		<h1><a href = "//sci-hub.wf/Find-DOI.html" target="_blank">How to quickly find the DOI number of an article</a></h1>
		<p>If you still cannot find thesis through DOI, we will include relevant articles as soon as possible,please try searching the corresponding DOI again after a while.</p>
		<div id = "message">&#9432; you can close this page and check later if the article has been downloaded</div>
                        <div id = "noproxy">no matching proxies found</div>
                        <div id = "found">proxy found, please wait</div>
<br></br>
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-7696326278603752"
     crossorigin="anonymous"></script>
<ins class="adsbygoogle"
     style="display:inline-block;width:970px;height:90px"
     data-ad-client="ca-pub-7696326278603752"
     data-ad-slot="4246281558"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
<p></div>
<script>var allurl=window.location.href;window.history.pushState({}, 0, "https://" + window.location.host);</script>
<script>setTimeout(function() {  window.history.pushState({}, 0, allurl);  }, 1000);</script>
    </div>
    <a id ="back" href = "/">&larr; return to main</a>
</body>
</html>

It does however seem to actually be able to find the paper and save with a reasonble name, i.e. [first author][year].
I would delve into the code to see where it is going wrong, but my skills are mainly limited to basic python.

Cheers!

"Conditional binary operator expected"

Mac OS 12.0.1

/usr/local/bin/scitopdf: line 21: conditional binary operator expected
/usr/local/bin/scitopdf: line 21: syntax error near `READER'
/usr/local/bin/scitopdf: line 21: `[[ -v READER ]] || \'

This happens just running the script - with or without options.

search without crossref

I there the possibility to look up a pdf just on the basis of its title?

When I search for:
scitopdf "The mirror-neuron system: A Bayesian perspective":

▄▀▀░▄▀▀░█░▀█▀░▄▀▄▒█▀▄░█▀▄▒█▀
▄██░▀▄▄░█░▒█▒░▀▄▀░█▀▒▒█▄▀░█▀
User search not found on Crossref.
Paper not found.

... no results are found. When I search with the same string/title on google scholar or scihub I find exactly the one paper I want.
Only crossref does not seem to know what I am looking for.

Is there an option to disable crossref for your program?

I am using the Arch AUR version of scitopdf (r157.8d4d2b8-1).