foxite / patreondownloader

A command line tool that downloads all media from a Patreon campaign.

PatreonDownloader

A simple command line tool that downloads all media from a Patreon campaign.

It's not exactly easy to use, but it's explained below.

Go here to download a build.

Usage

The program needs two pieces of information:

• The link to the /api/posts page for the creator you want to download the content of. This tells the program where to look for posts.
• Your session token. This allows the program to download everything you have access to. The program can automatically extract this cookie from supported browsers, or you may enter it yourself.

Do not take this lightly. Your session token can be used to gain full access to your account. As such, I encourage healthy paranoia when using this program, which is why the full source code is available. Read the Security heading below for more details.

To actually get the information:

• Link to /api/posts: Go to patreon.com and visit the posts page of the creator. Using your browser's developer tools (press F12 in most browsers), open the network inspector, and refresh the page. You will see it loading several dozen pages. Find the one that starts with "posts?". The question mark is important. Depending on your browser, it may also start with "/api/posts?". The full page address + query will be quite long. This is the string we need first.
• Session token, if you opt to enter it yourself: Also on patreon.com, on any page, get the value of the cookie named "session_id". This differs per browser, but for Chromium-based browsers it can be done by clicking on the HTTPS icon next to the page address. On Firefox, you need to view the Storage inspector in your F12 menu.

Once you have the information you can start the program, and give it the information.

The first time you run it, it will create a folder in your documents folder called PatreonDownloader, and within it, a file called posts.json, which contains the full data of all posts it downloads. At this point it does not yet download media. It is limited to downloading 20 posts at a time, but every page will be added immediately to the file, ensuring safety in the case of an unexpected crash. It will wait ten seconds between each download - this is to prevent your IP getting blocked because you are exceeding the rate limit.

The next time you run it, it will ask you what you want to do with the posts.json file. If you decide to download all images, it will create folders inside Documents/PatreonDownloader for every post, and within each folder, all media that has been downloaded for that post. Each folder will be named according to the date of the post, in YYYY-MM-DD format, and will also include the title of the post. The subfolders will contain the original files that were published, named as they were by the creator.

External media

The program will also attempt to extract links from posts, and if possible, download the files stored at those links. Currently, only Dropbox links are supported. If you want to download external media from other hosting sites, feel free to submit a PR.

Security

This program can automatically extract cookies from your browser, which is (correctly) identified as suspicious behaviour by certain antivirus programs. This is why the full source code is available. You can see that only the patreon.com session_id cookie is extracted and only used to authenticate the program with Patreon, and is not sent anywhere else. While the exact URL that the cookie is sent to is based on user input, the code will never send the cookie to any of my servers. Your cookies are safe.

View VirusTotal reports for version 1.0 here (Windows zip file) and here (Windows executable).

Contributing

Please stick to existing code conventions when contributing.

To add support for additional external hosting sites, look in LinkScraping to see how it works. When you have a working downloader, add it to the array in Program.DownloadMedia().

To add support for additional browsers to extract cookies from, look in CookieExtraction to see how it works. When you have a working extractor, add it to the array in Program.Main().

patreondownloader's Issues

Error: Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.

I pasted my session_id (not sure if that's correct) and also tried not putting anything for the token, but nothing worked. the link I pasted was to the /posts page as I want to bulk download all content

Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
   at bool Newtonsoft.Json.JsonTextReader.ParseValue()
   at bool Newtonsoft.Json.JsonReader.ReadAndMoveToContent()
   at bool Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, bool hasConverter)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, bool checkAdditionalContent)
   at object Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at object Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at object Newtonsoft.Json.JsonConvert.DeserializeObject(string value, Type type, JsonSerializerSettings settings)
   at T Newtonsoft.Json.JsonConvert.DeserializeObject<T>(string value, JsonSerializerSettings settings)
   at void PatreonDownloader.Program.DownloadAllPosts(HttpClient client, List<PostPage> pages, string backupFile) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 233
   at void PatreonDownloader.Program.Main(string[] args) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 116

When trying to download I get the following error that says I should forward to you.

Error: An unknown error has occured. Details have been saved in Documents/PatreonDownloader/error.log. Please forward it to the developer.

The error.log is bellow:

To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List<T>) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
Path 'data[6].relationships.audio.data.id', line 1, position 21404.
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at bool Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, object target)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at bool Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, object target)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at bool Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, object target)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, object existingValue, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at bool Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, object target)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, string id)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, object existingValue)
   at object Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, bool checkAdditionalContent)
   at object Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at object Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at object Newtonsoft.Json.JsonConvert.DeserializeObject(string value, Type type, JsonSerializerSettings settings)
   at T Newtonsoft.Json.JsonConvert.DeserializeObject<T>(string value, JsonSerializerSettings settings)
   at void PatreonDownloader.Program.DownloadAllPosts(HttpClient client, List<PostPage> pages, string backupFile) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 233
   at void PatreonDownloader.Program.Main(string[] args) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 116```

Not Downloading from Dropbox

I scraped the profile of a creator who posts all her videos on patreon and the pictures on patreon and imgur...
the pictures worked fine (except imgur ofc)...

But i also noticed that no media was downloaded from dropbox...

I get error everytime i try to download a file

whenever i start to download a post, it says: System.ArgumentNullException: Value cannot be null. (Parameter 'html')
at HtmlAgilityPack.HtmlDocument.LoadHtml(String html)
at PatreonDownloader.Program.DownloadMedia(HttpClient client, CookieContainer cookies, List`1 posts) in E:\Development\PatreonDownloader\PatreonDownloader\Program.cs:line 167
at PatreonDownloader.Program.Main(String[] args) in E:\Development\PatreonDownloader\PatreonDownloader\Program.cs:line 83

can you msg me on discord with a potential fix? Duddy#0028

ALSO I get all errors I try to download a file

Hi, Foxite.
I'm having a problem downloading, similar to a previous one that was solved by you,
Every time I run the program it shows the following error:

System.NullReferenceException: Object reference not set to an instance of an object.
at void PatreonDownloader.Program.DownloadMedia(HttpClient client, CookieContainer cookies, List posts) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 166
at void PatreonDownloader.Program.Main(string[] args) in /mnt/data/Development/PatreonDownloader/PatreonDownloader/Program.cs:line 104

I even downloaded Chrome (which I particularly hate) but nothing worked.
I don't know how to proceed, I'm quite a layman with all this, sorry.
Could you please help me?

Exception when getting posts

I get this exception when trying to get posts. Maybe something has changed?

System.Collections.Generic.KeyNotFoundException: The given key 'post_tag' was not present in the dictionary.
at System.Collections.Generic.Dictionary2.get_Item(TKey key) at PatreonDownloader.CustomStringEnumConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in C:\Users\Foxite\source\repos\PatreonDownloader\PatreonDownloader\CustomStringEnumConverter.cs:line 26 at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings) at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings) at PatreonDownloader.Program.DownloadAllPosts(HttpClient client, List1 pages, String backupFile) in C:\Users\Foxite\source\repos\PatreonDownloader\PatreonDownloader\Program.cs:line 202
at PatreonDownloader.Program.Main(String[] args) in C:\Users\Foxite\source\repos\PatreonDownloader\PatreonDownloader\Program.cs:line 93

Cross-platform issues

This app was mostly developed on Windows but since I started using Arch (btw) I have found that:

• It can't detect a Firefox installation (and probably not a Chrome installation either)
• Since BouncyCastle uses .NET Framework, it probably can't decrypt Chrome's cookies

These problems likely exist on Mac as well.

Feature request: ability to use different directories for each creator

Thanks a lot for your program!

I just discovered it and he is currently saving my life archiving about 60 pages from an creator, after I tried doing it by hand...

However, since it is harcoded to use the PatreonDownloader directory for everything, in practice it means it is not possible to use it to download multiple campaigns without renaming/swapping this folder between each one.

Would it be possible in a future version to add the possibility to either work in different PatreonDownloader subfolders depending on the creator, or add a way to specify a custom working directory?