On disk cache instead of in memory cache about alamofireimage

[cnoon]

I'm going to leave this issue open for a while for the community to +1 if they'd like to see it implemented.

[gringoireDM]

Hello guys. I opened a pull request to add this feature #278

[cnoon]

Yup. They already are cached in the NSURLCache stored in the session configuration inside the session of the image downloader. Man that's a mouthful.

The thing is though that they don't technically need to be. You could disable the NSURLCache and only use the persistent image cache if you wanted. Again, the main issue at that point is how to replicate the cache policy behavior of NSURLCache. IMO, you wouldn't want to try. There are a bunch of different header values to take into consideration.

Now another way to look at it would be that it wouldn't respect cache policy headers. It would be on the client to remove the images if they wanted to replace them. The persistent cache could also have some default duration until they were purged. Maybe a week? Maybe a month? Maybe an enum with a few sensible choices?

To also add to the fire, if you leverage the NSURLCache for the persistent behavior, you'll end up re-running your image filters every time you re-launch your app. The thing to keep in mind is that the UIImageView and the ImageDownloader only searches in the ImageCache for the cached image. It doesn't look into the NSURLCache. This means that if the image is cached in the NSURLCache but not in the Image Cache and does NOT use an ImageFilter, then the request is going to have to fire before the image is dug out of the NSURLCache. The upside or downside of this is that your ImageTransition is going to run in this case which may not be what you want because it will be SUPER fast.

On the flipside, this may be what you want. Tough to say. Also, you don't know if the image data is in memory or on disk for the NSURLCache. You probably want to keep this operation async in case the image data isn't loaded into memory yet. Think if you ended up pulling the data off disk on the main thread for hundreds of images in a scroll view. Sounds problematic. 😕

[cnoon]

Thank you everyone for all the feedback here. I really appreciate everyone's thoughts to help us shape our strategy. Both @kcharwood and I have decided that we will NOT be directly supporting a custom on-disk cache in AlamofireImage. There are too many different expectations of how it should work and overall just makes caching much more complicated for the general user.

With that said, the design of the ImageCache protocol allows you to create your own implementation of an on-disk image cache if you would like. AlamofireImage is already designed to allow this.

Thanks again! 🍻

[cnoon]

It's definitely a card in our Trello board. I didn't put it together yet because I wasn't exactly sure what the best way was to handle cache expiration timestamps for each image. I actually assumed someone in the community would end up requesting this feature and then we could discuss the implementation in that issue. And go figure, here we are!

I think the best way to proceed would be to take feature requests here in this issue from everyone to see how they would like it to work. Obviously yourself included @mattdelves 😉

Once we get some feedback, I can put together an actual feature list and I'll get one created.

[kcharwood]

Can those images be cached using the URL loading system?

[tkrajacic]

+1 ;)

[25matt12]

+1 :)

[kingslay]

+1 :)

[rosskimes]

+1

[trasch]

+1

[DenisLaboureyras]

+1

[iliaskarim]

+1

[markkrenek]

+1, I was just struggling with this. I have never-changing image URLs but on each app launch, the images are pulled down again from the remote URL

[cnoon]

@markkrenek for now you should at least be able to leverage the NSURLCache to not have to hit the network each time. This doesn't avoid having to re-run the ImageFilter over the image each time if necessary, but at least you can avoid hitting the network.

[loudmouth]

+1

[jshier]

👍

Having an on disk cache we control would avoid issues like #22.

[Lweek]

+1
@cnoon I would let developers enable HTTP ETag cache verification. With ETags server is in control of cache. Plus maybe some method that will let me manually purge images that are older than given NSDate parameter. Just for cleaning purposes. Automatic purging would be problematic. Maybe some kind of "lock" mechanism which will let programmers choose if file is dirty or clean cache (e.g. purgeable).

[kcharwood]

I'm still not super clear what type of functionality we could provide here outside of NSURLCache. If you simply want to "force" the image to be cached, you can use setDataTaskWillCacheResponseBlock: on the manager. I'm not super concerned with having to rerun the image filters.

What extra type of functionality would you want to see out of a custom on disk cache?

[cnoon]

You make very valid points @kcharwood. That's exactly why I didn't build one in the first version. Here's a quick pros/cons list as I see it:

Pros

• Can cache multiple versions of the same remote image directly on disk

  This could help you minimize your cache storage space if you are always downscaling all the remote images. Probably a less common case, but just thinking out loud.

• Won't have to re-run ImageFilters

  Obviously not important for super fast filters, but for filters leveraging CoreImage filters or composite filters, this could be much faster

• Wouldn't run the ImageTransitions in the UIImageView because it could dig the images directly out of the cache

  This could be debated as well. The image transition should probably run if the image isn't already in-memory, otherwise you'll run into inflation hiccups and stutters. I think I just talked myself into this not being a Pro.

Cons

• You have to manually delete your cached files if you want to replace them
• Will not respect the same caching controls as NSURLCache
• You would have to remember to set your NSURLCache to nil on the URL session configuration or you would be double caching on disk

I'm sure I missed a few things, but that's certainly a good start.

[cnoon]

I like the eTag idea @Lweek. I'm still not exactly sure how that would be implemented at both the ImageDownloader and UIImageView levels, but it's certainly a good idea. Just need to think through the use cases. Additionally, the caching system cannot get too complex because I don't want to have to end up tracking so much data that we need to use a database. I'd prefer to not have to write a SQLite wrapper in this library 😉

[iliaskarim]

@cnoon & @kcharwood I'd like to believe that NSURLCache provides the functionality that most people are requesting here as you suggest.

Which is why I am all the more perplexed by the following line from the README:

If you do not use image filters, it is advised to set the memory capacity of the NSURLCache to zero to avoid storing the same content in-memory twice.

Doesn't this disable the NSURLCache altogether?? Let me know if I've misunderstood.

[jshier]

@iliaskarim The memory and disk capacity of NSURLCache are separate, so you can can shut one off while keeping the other.

@cnoon I've given your Cons a bit of thought.

• You have to manually delete your cached files if you want to replace them

I'm not sure this is a common use case. If I've got to the trouble of turning the (hypothetical) ImageCache disk setting on, presumably I want to download the images once and manually control when they should be removed (e.g. at user logout, etc.). In addition, offering simple timeouts or other caching rules for the disk cache is possible, if complex.

• Will not respect the same caching controls as NSURLCache

Two things. First, this may be exactly what users want, as one of the primary reasons to use a separate disk cache is to work around crappy backends that don't properly mark their images as cacheable, or otherwise have bad cache settings. Second, it seems like there should be a way for such a cache to take direction from the underlying networking, so that images are replaced on disk only if the network says they should be. This scenario is likely most useful for users who want to cache the result of expensive filters or resizing. But again, it's complex.

• You would have to remember to set your NSURLCache to nil on the URL session configuration or you would be double caching on disk

True, but Alamofire has always been pretty good about documentation.

All of that said, I think many of the use cases for a dedicated on disk cache could be taken care of by exposing more control of the underlying NSURLCache, without forcing developers to build their own NSURLSession, or by exposing a shouldCache block at a higher level than overriding the closure. That way users could work around one of the primary uses for a disk cache, a poor backend that isn't configured for caching, leaving just the users of heavy filtering and resizing to be served.

In the end, I think enhancing the cache controls visible to the user would allow a disk cache, if implemented, to be greatly simplified, so perhaps a combination of both approaches would be in order. Of course, this is just a theoretical approach, so I don't know how realistic any of this solution would be.

[iliaskarim]

thanks for clearing up my misunderstanding, @jshier

I agree with you that one of the primary reasons to use a separate disk cache is to work around crappy backends that don't properly mark their images as cacheable, or otherwise have bad cache settings. I am not sure whether or not that justifies building this feature, though. My intuition tells me it does not, since the real solution is for developers to fix their server's HTTP header cache settings.

Perhaps we could instead provide links to resources explaining HTTP cache-control settings and how they interact with NSURLCache.

[jshier]

Unfortunately, it's not always possible for those of us stuck building apps for clients with their own backends to get changes made. 😭

[kcharwood]

@jshier In the case where you want a custom cache, what would be your expected logic be for evicting the cache?

[jshier]

@kcharwood I think the most common case, where the user wants automatic eviction at all, would be some sort of static expiration time. I've used EGOCache before which supports storing items in a cache with an expiration time, usually to match whatever the expected backend update interval is (e.g. the menu for restaurant chain will only change every 2 weeks at most) to store items for that long. Of course, a maximum disk capacity or a general maximum age could be used to do automatic eviction.

Of course, that's just for the cache itself. Some users may want auto eviction based on server responses as well.

[kimdv]

@cnoon I have implemented

imageView.af_setImageWithURL(URL, placeholderImage: nil, imageTransition: .CrossDissolve(0.25))

But when i use the images later it downloads the images again and it don't work if my app is offline. Is there a way to save the downloaded images from the web in my app?

[cnoon]

If your app is offline, you could try to use the NSURLRequest. NSURLRequestCachePolicy.ReturnCacheDataElseLoad cache policy if you cache headers allow it. Otherwise you would be best off using Alamofire directly with download requests to save the images directly to disk.

[kimdv]

@cnoon if I've understood the .UseProtocolCachePolicy correctly, it should ask for revalidation of data, like the images here.

This is the response i get when i fetch an image, and it expires next year, so it should reload from cache, if i have understood the .UseProtocolCachePolicy correctly?

This is my header:

code: 200, headers {
    "Accept-Ranges" = bytes;
    "Cache-Control" = "public, max-age=31536000";
    "Content-Length" = 544924;
    "Content-Type" = "image/jpeg";
    Date = "Thu, 17 Dec 2015 08:08:17 GMT";
    Etag = "\"4201af61c6e0b8330cb368b884fa2edd\"";
    Expires = "Fri, 16 Dec 2016 08:08:17 GMT";
    "Last-Modified" = "Sun, 28 Sep 2014 00:00:58 GMT";
    Server = "cloudflare-nginx";
    Vary = "Accept-Encoding";
    "cf-cache-status" = MISS;
    "cf-ray" = "256123a2b01e2b9a-AMS";
    "x-amz-id-2" = "KAIX7Eez+GnFg1q/opuHpVV9KsBWBDVB86i567rPNfZbIpRMtJvCq4Bxwu+GNL2x";
    "x-amz-request-id" = 7F2B4B5AA837622E;
} }

[cnoon]

That's mostly correct @kimdv. I'm not entirely sure about your understanding on .UseProtocolCachePolicy. I'm not sure if it will pull from the cache if the device is offline, or just return you a connectivity error. You'll have to test this theory.

I am however confident in the behavior of .ReturnCacheDataElseLoad and .ReturnCacheDataDontLoad. Those will certainly load from the cache by default, and then .ReturnCacheDataElseLoad will hit the network if necessary. Problem is that this ignores the state of the Expires header.

So in summary, I'd say .UseProtocolCachePolicy could be exactly what you want if it does in fact not throw an error in an offline state, but instead returns you the cached image. You also need to keep in mind then that the NSURLCache is only so big. You should always configure a custom one tailored to your own needs.

[cnoon]

You should also read through this article if you haven't already.

[ayking]

+1

[sahilanguralla]

+1

[nemezis16]

+1

[moto0000]

+1

[piasetskiy]

+1!