noppesthefolf / e621client

An unofficial .NET Standard 2.1 wrapper for the e621 API

E621Client

E621Client is an unofficial .NET Standard 2.1 library for interacting with the e621 and e926 API maintained by me, Noppes. It has support for nullable and non-nullable reference types.

Table of Contents

• E621Client
  • Table of Contents
  • Completeness
  • Installation
  • Getting started
  • Authentication
    • Logging in
    • Logging out
  • Functionality per API area
    • Posts
      • Retrieving a post
      • Retrieving posts
        • Without navigation
        • Navigation using pagination
        • Navigation using relative positioning
    • Tags
      • Retrieving a tag
      • Retrieving tags
        • Without filter
        • Using tags and their names
        • Using a search query
    • Pools
      • Retrieving a pool
      • Retrieving pools
    • Users
      • Retrieving a user
      • Retrieving the logged-in user
    • Favorites
      • Adding a post
      • Removing a post
      • Retrieving favorites
    • Artists
      • Retrieving an artist
      • Retrieving artists
    • Database exports
    • IQDB (Reverse image searching)
    • Additional
      • Get response body as a stream
  • Testing
  • Report a bug
  • Contributing

Completeness

This project is far from a complete wrapper around the e621 API. A table can be seen below to give you a quick idea of how complete this library is. I might never implement most of the API because I'd never use those endpoints myself. However, feel free to implement those areas yourself if you need them! Just make sure the check out how to contribute first.

Legend

Symbol	Meaning
❌	Not implemented
➖	Partially implemented
✔️	Fully implemented

Cover per API area

Area	Complete	Comment
Authentication	✔️
Posts	➖	Only the retrieval of posts
Tags	➖	Only the retrieval of tags
Tag aliases	❌
Tag implications	❌
Notes	❌
Pools	➖	Only the retrieval of pools
Users	➖	Only the retrieval of a user by name
Favorites	✔️
Artists	➖	Only the retrieval of artists, not documented by e621
IQDB	✔️	Not yet documented by e621 at the moment of writing
DB export	➖	All except wiki pages

Installation

E621Client is available as a NuGet package listed as Noppes.E621Client. You can easily install it using either the Package Manager Console or the .NET CLI.

Package Manager Console

Install-Package Noppes.E621Client -Version 0.8.2

.NET CLI

dotnet add package Noppes.E621Client --version 0.8.2

Getting started

You will need a IE621Client instance in order to interact with the API. These instances can only be created using the E621ClientBuilder class. The builder will allow you to create your very own personalized IE621Client instance in a fluent manner based on the specific needs of your application. Just make sure you at least specify User-Agent information, as e621 requires it. Not specifying it will cause an exception to be thrown.

Bare minimum example

var e621Client = new E621ClientBuilder()
    .WithUserAgent("MyApplicationName", "MyApplicationVersion", "MyTwitterUsername", "Twitter")
    .Build();

However, you might need something a little more suited for your application. The default IE621Client instance built above uses settings tuned to make sure the load on e621's side is kept to a minimum. This is not desirable if you're, for example, developing an interactive tool and therefore want it to be as snappy as possible.

Example for an interactive application

var e621Client = new E621ClientBuilder()
    .WithUserAgent("MyApplicationName", "MyApplicationVersion", "MyTwitterUsername", "Twitter")
    .WithMaximumConnections(E621Constants.MaximumConnectionsLimit)
    .WithRequestInterval(E621Constants.MinimumRequestInterval)
    .Build();

IE621Client instances can be disposed of, but you generally want to treat an IE621Client instance as a singleton. It uses a HttpClient behind the scenes which gets disposed when you dispose the associated IE621Client. You can read more about why that's bad at You're using HttpClient wrong and it is destabilizing your software if you're interested.

Authentication

Authentication will have to take place in order to be gain access to certain protected resources. E621Client makes this convenient by mimicking a login- and logout-based system.

Logging in

Logging in can only be done based on a combination of username and API key.

Log in using a username and API key

bool success = await e621Client.LogInAsync("MyUsername", "MyApiKey");

The method returns a boolean that indicates whether not the login attempt was a success. It does this by requesting the user's favorites. This method executing successfully, implies that the user entered valid credentials. In case the method return false, it may also mean that the user doesn't have API access enabled in their account. There is no way of determining if that's the case though.

Logging out

You need to log out in order for another log in to be allowed to happen. You'll most likely don't need this this for most applications, but here is a method that does so anyway.

Log the currently logged-in user out

e621Client.Logout();

Functionality per API area

The main point of this section is to show how the e621 API endpoints map to E621Client methods. The methods themselves are sometimes more elaborately documented than they are here. So don't worry if things may still seem kind off vague after reading this, check the method documentation out too!

Posts

Retrieving a post

A post can be retrieved using either its ID or its image's MD5 hash by calling the GetPostAsync method.

Retrieving a post by its ID

var postById = await e621Client.GetPostAsync(546281);

Retrieving a post by its image's MD5 hash

var postByMd5 = await e621Client.GetPostAsync("900e98af5b512ba1a5f8a1a9885c1ef1");

Retrieving posts

A collection of posts can be retrieved using the GetPostsAsync method. There is a limit on how much posts can be retrieved in a single call to this method. This limited is defined at E621Constants.PostsMaximumLimit. Therefore, you need to be able to navigate through e621's post collection. There are two ways to do this: pagination and relative positioning.

Without navigation

You can request a bunch of posts without navigation, but the use cases are very limited of course.

Retrieve a collection of posts tagged "canine" with the maximum number of posts retrievable in a single call without using any navigation

var posts = await e621Client.GetPostsAsync("canine", limit: E621Constants.PostsMaximumLimit);

Navigation using pagination

You're probably already familiar with the concept of pagination: a collection of something, in this case posts, is split into parts of equal size. Each of those parts is assigned a number and these parts can then be requested using that number. That number is which we call a "page".

Be aware that there is a limit on the maximum allowed page number. This limit is defined at E621Constants.PostsMaximumPage. Exceeding this number will cause an exception to be thrown.

Retrieve the fifteenth page of a collection of posts tagged "canine"

var posts = await e621Client.GetPostsAsync("canine", 15);

Retrieve the fifteenth page of a collection of posts tagged "canine" with the maximum number of posts retrievable in a single call

var posts = await e621Client.GetPostsAsync("canine", 15, E621Constants.PostsMaximumLimit);

Navigation using relative positioning

This may sound a bit scary at first, but it really isn't. All you need to do is specify both an post ID and a position. The position parameter defines the position of the returned posts relative to the given post ID.

Let's take post with ID 1000 as an example. Passing this ID in combination with Position.Before will cause the posts 999, 998, 997, etc. to be retrieved. Using Position.After will retrieve the posts 1001, 1002, 1003, etc. You should use this method if you don't need pagination or need to avoid the limit pagination comes with. Moreover, this is the most efficient way to navigate through posts.

Retrieve a collection of posts with an ID less than 200

var posts = await e621Client.GetPostsAsync(200, Position.Before);

Retrieve a collection of posts with an ID greater than 200

var posts = await e621Client.GetPostsAsync(200, Position.After);

Retrieve a collection of posts tagged "canine" with an ID greater than 200 using the maximum limit of posts retrieved in a single call

var posts = await e621Client.GetPostsAsync(200, Position.After, "canine", E621Constants.PostsMaximumLimit);

Tags

Retrieving a tag

A tag can either be retrieved by either its name or ID by using the GetTagAsync method.

Retrieve a tag by ID

var tag = await e621Client.GetTagAsync(813847);

Retrieve a tag by name

var tag = await e621Client.GetTagAsync("noppes");

Retrieving tags

There are a couple of different ways a collection of tags can be retrieved: without using any filter, using the names of the tags and using a query.

Searching for tags without using filter, opens up the usage of pagination with relative positioning. This works in exactly the same way as it does for posts.. All of the other available overloads make use of pagination as you're used to.. The maximum allowed page number is defined at E621Constants.TagsMaximumPage.

Note that there is a limit on the number of tags that can be retrieved in a single call to any of the overloads. This limit is defined at E621Constants.TagsMaximumLimit.

Without filter

The first way of retrieving tags is without using any specific name filter with the GetTagsAsync method.

Get the maximum possible number of tags after ID 1000

var tags = await e621Client.GetTagsAsync(1000, Position.After, E621Constants.TagsMaximumLimit);

Get the 10th page using the maximum possible number of tags ordered by the number of posts making use of the tag

var tags = await e621Client.GetTagsAsync(10, E621Constants.TagsMaximumLimit, order: TagOrder.Count);

Using tags and their names

A collection of tags can also be retrieved using their names using the GetTagsUsingNamesAsync method.

Get tags using their names

var tagNames = new []
{
    "noppes",
    "blep",
    "fur"
};
var tags = await e621Client.GetTagsByNames(tagNames);

Using a search query

Tags can also be retrieved using a search query on the tag their names using a wildcards, for example, with an overload of the GetTagsUsingNamesAsync method.

Get the first page of tags that start with 'wolf' in the species category

var tags = await e621Client.GetTagsByNames("wolf*", category: TagCategory.Species);

Pools

Retrieving a pool

You can retrieve the information of a single pool by its ID. If there exists no pool with the given ID, a null value will be returned. If you need to retrieve more than one pool, you should take a look at the next section of the documentation. There is a more efficient way to do that than to call this method multiple times.

Retrieve information about pool with ID 621

var pool = await e621Client.GetPoolAsync(621);

Retrieving pools

As an expansion upon the previous section, you can retrieve multiple pools by their IDs. IDs to which there is no pool associated, will be left out of the result.

Retrieve pool with ID 621 and pool with ID 926

var poolIds = new[]
{
    621,
    926
};
var pools = await e621Client.GetPoolsAsync(ids: poolIds);

You can also get a listing of all the pools on e621. You can navigate through all of the available pools using either pagination or relative position. Both of these concepts have already been explained in the posts section of the documentation and therefore won't be further elaborated on here. Using relative positioning won't allow you to specify an order in which pools should be sorted.

Get the third page of pools with the response containing as much pools as allowed

var pools = await e621Client.GetPoolsAsync(3, limit: E621Constants.PoolsMaximumLimit);

To narrow down the pools returned by e621, you can filter based on the following attributes of a pool: name, description, by who it was created, whether its still being actively updated, whether it has been deleted and its category.

Get (the first page of) pools of which the name ends in "cat" and which are not deleted, in descending order by the amount of posts contained in the pool.

var pools = await e621Client.GetPoolsAsync(name: "*cat", isDeleted: false, order: PoolOrder.PostCount)

Users

Currently E621Client doesn't support much of the users area of the API mainly due to there being no documentation on it whatsoever at the moment this was written.

Retrieving a user

You can retrieve a part of the info available about a user by searching for them using their username.

Get information about a user by searching by their username

var user = await e621Client.GetUserAsync("noppes");

Retrieving the logged-in user

You can also retrieve the same info for the user that is currently logged-in.

Get information about the currently logged-in user

var user = await e621Client.GetLoggedInUserAsync();

Favorites

Adding a post

Adding a post to the logged-in user their favorites, can be done with the AddFavoriteAsync method.

Adding a post using its ID to the logged-in user's favorites

await e621Client.AddFavoriteAsync(546281);

Removing a post

You can remove a post from the logged-in user their favorites using the RemoveFavoriteAsync method.

Removing a post by its ID from the logged-in user's favorites

await e621Client.RemoveFavoriteAsync(546281);

Retrieving favorites

We can retrieve the posts favorited by either the logged-in user or some other user by using their user ID.

Retrieving the first page of posts favorited by the logged-in user

var favorites = await e621Client.GetOwnFavoritesAsync();

Retrieving the fifth page of posts favorited by the logged-in user

var favorites = await e621Client.GetOwnFavoritesAsync(5);

Retrieving the first page of posts favorited by the user with ID 11271 (SnowWolf)

var favorites = await e621Client.GetFavoritesAsync(11271);

Retrieving the seventh page of posts favorited by the user with ID 11271 (SnowWolf)

var favorites = await e621Client.GetFavoritesAsync(11271, 7);

Artists

Retrieving an artist

You can retrieve a single artist by their ID.

Retrieve information about artist with ID 8695, which is the artist WagnerMutt

var wagnermutt = await e621Client.GetArtistAsync(8695);

Retrieving artists

You can also retrieve a list of artists and optionally filter the results by name, for example. Pagination works the same way as it does for other endpoints with normal pagination using numbers and relative positioning.

Retrieve a list of artists that contain the the name "wagner"

var wagners = await e621Client.GetArtistsAsync(name: "wagner")

Database exports

e621 provides exports of posts, pools, tags, tag aliases, tag implications and wiki pages.

E621Client supports downloading and the reading all of these exports except wiki pages. Due to this functionality being unlikely to be used in combination with it introducing a few dependencies, it has its own separate package: Noppes.E621Client.DbExport.

Package Manager Console

Install-Package Noppes.E621Client.DbExport -Version 0.8.2

.NET CLI

dotnet add package Noppes.E621Client.DbExport --version 0.8.2

After installing the package, you need to get a database export client. You can get one by calling the GetDbExportClient method on your IE621Client instance, as shown below.

Get a database export client

var dbExportClient = e621Client.GetDbExportClient();

The next step you want to take whenever you want to retrieve a database export, is get a list of all the database exports available on e621. This can be done using the GetDbExportsAsync method on your database export client. This list can also be viewed on e621.

Get a list of all the available database exports

var exports = await dbExportClient.GetDbExportsAsync();

Now that you have a list of all the database exports available on e621, it is time to choose which one you'd like to download and read. The retrieved exports contain a handy extension method named Latest. With this method, you can get the latest database export of a given type (posts, pools, tags, etc.).

After selecting the desired database export, you need to get the data as a stream using the GetDbExportStreamAsync method. This stream, depending on what export you're downloading, can then be read using one of the following methods on your database export client:

Posts: ReadStreamAsPostsDbExportAsync
Pools: ReadStreamAsPoolsDbExportAsync
Tags: ReadStreamAsTagsDbExportAsync
Tag implications: ReadStreamAsTagImplicationsDbExportAsync
Tag aliases: ReadStreamAsTagAliasesDbExportAsync

The code below shows a concrete example of how the use the interface described above.

Print the IDs of the posts in the latest post database export to the console

var export = exports.Latest(DbExportType.Post);
await using var stream = await dbExportClient.GetDbExportStreamAsync(export);
await foreach (var post in dbExportClient.ReadStreamAsPostsDbExportAsync(stream))
  Console.WriteLine(post.Id);

IQDB (Reverse image searching)

Note: Be sure to check out another project of mine, fluffle.xyz, if you're interested in this kind of functionality.

You can reverse search an image on e621 by either a locally stored image, a stream or a URL. In addition it is also possible to use another post as reference for the reverse search query (uses the image attached to the post). Reverse searching will return a collection of posts of which the images are similar to the submitted image/post. The returned posts have an additional property named IqdbScore which can be used to assess how similar the image is to the submitted one. E621Client will by default not return posts that have been deleted. However, if you'd like to include them, you can simply pass a boolean to any of the methods associated with querying IQDB.

In case you're using the URL method, note that e621 will download images only from domains whitelisted by them. Which domains are on the whitelist is unknown. You should test if the domains of the URLs you are planning to use are whitelisted or not.

Reverse image searching using a file, also returning deleted posts

var results = await e621Client.QueryIqdbByFileAsync("/my/path", false);

Reverse search the image of a post

var results = await e621Client.QueryIqdbByPostIdAsync(546281);

Reverse image searching using a URL

var results = await e621Client.QueryIqdbByUrlAsync("https://url.net");

Reverse image searching using a stream

// You can use any stream, a FileStream is simply used as an example here
await using var exampleStream = File.OpenRead("/my/path");
var results = await e621Client.QueryIqdbByStreamAsync(exampleStream);

Additional

Get response body as a stream

In case you need to get data from e621 that requires authorization, especially images, you can request said data as a Stream by using the GetStreamAsync method.

Get the response body as a stream from a given URL

await using var stream = await e621Client.GetStreamAsync("my/url");

Testing

E621Client supports testing by mocking. E621ClientBuilder.Build() will return an interface IE621Client which can be mocked using a mocking framework. You can use this to test your own logic with different responses of the IE621Client.

Report a bug

You can open an issue. Please make sure to include a piece of code that reproduces the bug. That will make troubleshooting a lot easier on my side! If you do not have a GitHub account, then also feel free to contact me.

Contributing

Contributions to this project are very welcome! Please open an issue where you describe what you'd like to contribute and why it is useful if that is not obvious. If you want to contact me personally, you can find places to contact me at on my website.

e621client's Issues

Add support for score_cutoff in IQDB searches

E621 allows you to set this when doing an IQDB search on their site, in my testing so far it seems like a value >= 80 means some correct matches are not returned. A value below 60 definitely returns some false positives. Around 75 seems to be a good value. The default when nothing is specified seems to be higher (90?) so some correct matches would not be returned. So being able to specify the cutoff would be quite useful.

Add support for usage with CancellationTokens

Adding an optional CancellationToken to each request, will allow it to be canceled. This shouldn’t be too much work and becomes especially interesting in combination with async enumeration.

Blazor WebAssembly support

Blazor on WebAssembly implements just parts of the functionality the HttpClientHandler class is expected to have. Part of this due to certain technical limitations the browser brings with it, the other part is because the Blazor team removed features they considered bloat in order to cut down the size of the assemblies that have to be shipped to the browser. Blazor is at the time of writing still in preview, but is expected to get a stable release in May 2020. It’d be nice to be able to support Blazor on WebAssembly in order to enable the development of PWAs using this library.

Allow mocking of the E621Client to facilitate unit testing

I ran into an issue when using your package to interact with the E621 API. I wanted to unit test the code that makes use of the E621Client

I have no real code examples to show but it boils down to this:
Because of the builder / factory pattern the constructor of E621Client is set to internal because of which a mocking framework cannot create a mocked instance of the client.

It would be a possibility to solve this by creating a IE621Client interface that gets returned by the builder so that we can mock on that interface.

I will try to implement this and will link a PR to this issue when it's ready.

Support has_notes and duration in Post/IqdbPost

Noticed these two are in the JSON data, but not exposed in the data types.
Interestingly enough, they aren't documented on the wiki either.

Simple change in Post.cs:

        /// <summary>
        /// Whether the post has notes associated with it.
        /// </summary>
        [JsonProperty("has_notes")]
        public bool HasNotes { get; set; }

        /// <summary>
        /// The duration of the post, if it is a video, or null otherwise
        /// </summary>
        [JsonProperty("duration")]
        public double? Duration { get; set; }

IqdbPost.cs is a bit more complicated. has_notes is not there, but we have another thing instead which we can use which does not exist in normal posts, namely last_noted_at:

        [JsonProperty("last_noted_at")]
        public DateTimeOffset? LastNotedAt;

        [JsonProperty("duration")]
        public double? Duration { get; set; }

And in AsPost():

                HasNotes = LastNotedAt != null,
                Duration = Duration

I've tested these changes and they seem to work fine and the behavior corresponds with what is expected. There is the possibility of a post having a note added and later deleted, resulting in last_noted_at being set even though the post does not currently have any notes, so IqdbPost wrongly assumes the post has notes. This is probably fairly rare though and not a high priority issue. It might not even be an issue, if last_noted_at is simply fetched from the latest note added rather than being a fixed field in the database.

On a side note, it is a shame that some of the post information is not available through the API.
Things like uploader name, deleted by, deleted reason, notes. Notes are used for translations and the usefulness of the others is self explanatory. These are not available through the database exports either (presumably they live in separate databases which are not exported)

Edit: Noticed Duration is there in DbExportPost, but gets converted to TimeSpan. So for consistency, best to do the same here, I suppose.

Add support for async enumeration

Some endpoints can be navigated using pagination or some other method. Adding methods that harness the power of the new IAsyncEnumerable introduced in C# 8, could be very convenient.

GetPostsAsync always returns empty with multiple tags

When using multiple tags, for example "domestic_cat" and "black_fur", separated by space the E621Client.GetPostsAsync method returns an empty list. It is however possible to search both tags separately and get results back.

Arbitrary tag limit?

The library appears to be hardcoded to only accept up to 6 tags in searches, but this does not seem to be an official API limitation. In fact, the API docs state "Any tag combination that works on the website will work here." and the website certainly does allow more than 6 tags. Is this limit arbitrary or am I missing something?