Coder Social home page Coder Social logo

readthis's Introduction

Gem Version Build Status Code Climate Coverage Status Inline Docs

Readthis

Readthis is a Redis backed cache client for Ruby. It is a drop in replacement for any ActiveSupport compliant cache and can also be used for session storage. Above all Readthis emphasizes performance, simplicity, and explicitness.

For new projects there isn't any reason to stick with Memcached. Redis is as fast, if not faster in many scenarios, and is far more likely to be used elsewhere in the stack. See this blog post for more details.

Rails 5.2+

Rails 5.2 and beyond has a Redis Cache built in. The built in Redis cache supports many of the same features as Readthis, as well as multi-tier caches and newer additions like cache key recycling.

Readthis is maintained for versions of Rails prior to 5.2 and new features will not be supported. If you are using Rails 5.2+ you should migrate to the built in Redis cache.

Footprint & Performance

See Performance

Installation

Add this line to your application's Gemfile:

gem 'readthis'
gem 'hiredis' # Highly recommended

Usage

Use it the same way as any other ActiveSupport::Cache::Store. Within a Rails environment config:

config.cache_store = :readthis_store, {
  expires_in: 2.weeks.to_i,
  namespace: 'cache',
  redis: { url: ENV.fetch('REDIS_URL'), driver: :hiredis }
}

Otherwise you can use it anywhere, without any reliance on ActiveSupport:

require 'readthis'

cache = Readthis::Cache.new(
  expires_in: 2.weeks.to_i,
  redis: { url: ENV.fetch('REDIS_URL') }
)

You can also specify host, port, db or any other valid Redis options. For more details about connection options see in redis gem documentation

Instances & Databases

An isolated Redis instance that is only used for caching is ideal. Dedicated instances have numerous benefits like: more predictable performance, avoiding expires in favor of LRU, and tuning the persistence mechanism. See Optimizing Redis Usage for Caching for more details.

At the very least, you'll want to use a specific database for caching. In the event the database needs to be purged you can do so with a single clear command, rather than finding all keys in a namespace and deleting them. Appending a number between 0 and 15 will specify the redis database, which defaults to 0. For example, using database 2:

REDIS_URL=redis://localhost:6379/2

Expiration

Be sure to use an integer value when setting expiration time. The default representation of ActiveSupport::Duration values won't work when setting expiration time, which will cause all keys to have -1 as the TTL. Expiration values are always cast as an integer on write. For example:

Readthis::Cache.new(expires_in: 1.week) # don't do this
Readthis::Cache.new(expires_in: 1.week.to_i) # do this

By using the refresh option the TTL for keys can be refreshed automatically every time the key is read. This is helpful for ensuring commonly hit keys are kept cached, effectively making the cache a hybrid LRU.

Readthis::Cache.new(refresh: true)

Be aware that refresh adds a slight overhead to all read operations, as they are now all write operations as well.

Compression

Compression can be enabled for all actions by passing the compress flag. By default all values greater than 1024k will be compressed automatically. If there is any content has not been stored with compression, or perhaps was compressed but is beneath the compression threshold, it will be passed through as is. This means it is safe to enable or change compression with an existing cache. There will be a decoding performance penalty in this case, but it should be minor.

config.cache_store = :readthis_store, {
  compress: true,
  compression_threshold: 2.kilobytes
}

Serializing

Readthis uses Ruby's Marshal module for serializing all values by default. This isn't always the fastest option, and depending on your use case it may be desirable to use a faster but less flexible serializer.

By default Readthis knows about 3 different serializers:

  • Marshal
  • JSON
  • Passthrough

If all cached data can safely be represented as a string then use the pass-through serializer:

Readthis::Cache.new(marshal: Readthis::Passthrough)

You can introduce up to four additional serializers by configuring serializers on the Readthis module. For example, if you wanted to use the extremely fast Oj library for JSON serialization:

Readthis.serializers << Oj

# Freeze the serializers to ensure they aren't changed at runtime.
Readthis.serializers.freeze!

Readthis::Cache.new(marshal: Oj)

Be aware that the order in which you add serializers matters. Serializers are sticky and a flag is stored with each cached value. If you subsequently go to deserialize values and haven't configured the same serializers in the same order your application will raise errors.

Fault Tolerance

In some situations it is desirable to keep serving requests from disk or the database if Redis crashes. This can be achieved with connection fault tolerance by enabling it at the top level:

Readthis.fault_tolerant = true

The default value is false, because while it may work for fetch operations, it isn't compatible with other state-based commands like increment.

Running Arbitrary Redis Commands

Readthis provides access to the underlying Redis connection pool, allowing you to run arbitrary commands directly through the cache instance. For example, if you wanted to expire a key manually using an instance of Rails.cache:

Rails.cache.pool.with { |client| client.expire('foo-key', 60) }

Differences From ActiveSupport::Cache

Readthis supports all of standard cache methods except for the following:

  • cleanup - Redis does this with TTL or LRU already.
  • mute and silence! - You must subscribe to the events /cache*.active_support/ with ActiveSupport::Notifications to log cache calls manually.

Like other ActiveSupport::Cache implementations it is possible to cache nil as a value. However, the fetch methods treat nil values as a cache miss and re-generate/re-cache the value. Caching nil isn't recommended.

Session Storage

By using ActionDispatch::Session::CacheStore it's possible to reuse :readthis_store or specify a new Readthis cache store for storing sessions.

Rails.application.config.session_store :cache_store

To specify a separate Readthis instance you can use the :cache option:

Rails.application.config.session_store :cache_store,
  cache: Readthis::Cache.new,
  expire_after: 2.weeks.to_i

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

readthis's People

Contributors

artofhuman avatar benlovell avatar dasnixon avatar dblandin avatar dependabot-preview[bot] avatar espen avatar juanperi avatar kagux avatar napolskih avatar rykov avatar sorentwo avatar tak1n avatar timcraft avatar tobinibot avatar workmad3 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

readthis's Issues

Incompatible character encodings: UTF-8 and ASCII-8BIT

Hello!
There is a problem when I try to save some russian symbols by russian key.

Here is an example to reproduce it:

it 'stores russian symbols by russian key' do
  cache.write('вгдмн', 'вгдмн')

  expect(cache.read('вгдмн')).to eq('вгдмн')
end

Stacktrace:

 1) Readthis::Cache#write stores russian symbols by russian key
     Failure/Error: store.set(namespaced, dumped)

     Encoding::CompatibilityError:
       incompatible character encodings: UTF-8 and ASCII-8BIT
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/connection/command_helper.rb:28:in `join'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/connection/command_helper.rb:28:in `build_command'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/connection/ruby.rb:374:in `write'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:271:in `block in write'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:250:in `io'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:269:in `write'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:228:in `block (3 levels) in process'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:222:in `each'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:222:in `block (2 levels) in process'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:367:in `ensure_connected'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:221:in `block in process'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:306:in `logging'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:220:in `process'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis/client.rb:120:in `call'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis.rb:754:in `block in set'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis.rb:58:in `block in synchronize'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis.rb:58:in `synchronize'
     # /bundle/2.3/gems/redis-3.3.2/lib/redis.rb:750:in `set'
     # ./lib/readthis/cache.rb:398:in `write_entity'
     # ./lib/readthis/cache.rb:106:in `block in write'
     # /bundle/2.3/gems/connection_pool-2.2.1/lib/connection_pool.rb:64:in `block (2 levels) in with'
     # /bundle/2.3/gems/connection_pool-2.2.1/lib/connection_pool.rb:63:in `handle_interrupt'
     # /bundle/2.3/gems/connection_pool-2.2.1/lib/connection_pool.rb:63:in `block in with'
     # /bundle/2.3/gems/connection_pool-2.2.1/lib/connection_pool.rb:60:in `handle_interrupt'
     # /bundle/2.3/gems/connection_pool-2.2.1/lib/connection_pool.rb:60:in `with'
     # ./lib/readthis/cache.rb:427:in `block in invoke'
     # ./lib/readthis/cache.rb:421:in `instrument'
     # ./lib/readthis/cache.rb:426:in `invoke'
     # ./lib/readthis/cache.rb:105:in `write'
     # ./spec/readthis/cache_spec.rb:38:in `block (3 levels) in <top (required)>'

My setup:

  • Readthis - 2.0.1 - master commit( 654fc11)
  • Ruby - ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
  • Redis - 3.2.4

Also on redis-store they had same problem and fixed it with commit shingara/redis-store@113d6e6.
Connected issue on redis-rb - redis/redis-rb#128.

Would it be a good solution here?
If not, what do you recommend to avoid this?

NotImplementedError: method 'div' called on terminated object"

Hello again.

In the last 12 hours, I've seen the following error about 10 times:

NotImplementedError: method `div' called on terminated object (0x007f948aef8028 flags=0x0 klass=0x0)
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/entity.rb:67 in new
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/entity.rb:67 in load
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/entity.rb:67 in load
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:71 in block in read
vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64 in block (2 levels) in with
vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63 in handle_interrupt
vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63 in block in with
vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60 in handle_interrupt
vendor/bundle/ruby/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60 in with
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:346 in block in invoke
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:341 in block in instrument
vendor/bundle/ruby/2.1.0/gems/activesupport-4.2.4/lib/active_support/notifications.rb:166 in instrument
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:341 in instrument
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:345 in invoke
vendor/bundle/ruby/2.1.0/bundler/gems/readthis-a774c6278aab/lib/readthis/cache.rb:68 in read

I am using the Oj serializer, and since it feels like a "use after free" condition, I figured it might be the C code in use by Oj, so I opened ohler55/oj#265.

I wanted to open this in case you might have encountered this and/or have any insight into debugging/resolving it. Unfortunately, I haven't been able to reproduce yet.

Cheers.

Set readthis connection pool after configuring cache_store as readthis_store?

If I configure this in my config/production.rb

  config.cache_store = :readthis_store, {
      expires_in: 30.seconds.to_i,
      namespace: 'cache',
      redis: { url: ENV.fetch('REDIS_URL'), driver: :hiredis }
  }

How can I set it's connection pool later when my puma workers call on_worker_boot

on_worker_boot is called after config/production.rb

The pool for readthis cannot be passed in via options

https://github.com/sorentwo/readthis/blob/master/lib/readthis/cache.rb#L59

I am stuck. Before I start hacking and making a mess I thought I'd ask here if I'm on the right track?

https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot

https://devcenter.heroku.com/articles/concurrency-and-database-connections#threaded-servers

How to close redis connection?

I use Readthis::Cache.new to get a client, but i can not find a way to close this redis connection. So I meet a very serious problem of redis, max number of clients reached.

Handle decimal expiration more gracefully, i.e. when SETEX expires_in.to_i is SETEX 0

Example from the ActiveModelSerializers caching benchmark:

Setting expire_in: 0.1 works fine using the ActiveSupport memory store

class CachingPostSerializer < ActiveModel::Serializer
  cache key: 'post', expires_in: 0.1, skip_digest: true

but readthis casts it to to_i #=> 0 which causes the below failure and stacktrace. (Perhaps Float(0.1).ceil would be preferable?)

/Users/benjamin/.rvm/gems/ruby-2.2.3/gems/readthis-1.2.1/lib/readthis.rb:26: warning: instance variable @fault_tolerant not initialized
I, [2016-05-30T23:57:14.753062 #37104]  INFO -- : Rendered CachingPostSerializer with ActiveModelSerializers::Adapter::Json (6.66ms)
I, [2016-05-30T23:57:14.753165 #37104]  INFO -- : Completed 500 Internal Server Error in 10ms
F, [2016-05-30T23:57:14.754827 #37104] FATAL -- :
Redis::CommandError (ERR invalid expire time in SETEX):
  redis (3.3.0) lib/redis/client.rb:121:in `call'
  redis (3.3.0) lib/redis.rb:769:in `block in setex'
  redis (3.3.0) lib/redis.rb:58:in `block in synchronize'
  /Users/benjamin/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
  redis (3.3.0) lib/redis.rb:58:in `synchronize'
  redis (3.3.0) lib/redis.rb:768:in `setex'
  readthis (1.2.1) lib/readthis/cache.rb:318:in `write_entity'
  readthis (1.2.1) lib/readthis/cache.rb:86:in `block in write'
  connection_pool (2.2.0) lib/connection_pool.rb:64:in `block (2 levels) in with'
  connection_pool (2.2.0) lib/connection_pool.rb:63:in `handle_interrupt'
  connection_pool (2.2.0) lib/connection_pool.rb:63:in `block in with'
  connection_pool (2.2.0) lib/connection_pool.rb:60:in `handle_interrupt'
  connection_pool (2.2.0) lib/connection_pool.rb:60:in `with'
  readthis (1.2.1) lib/readthis/cache.rb:346:in `block in invoke'
  readthis (1.2.1) lib/readthis/cache.rb:338:in `block in instrument'
  activesupport (4.2.6) lib/active_support/notifications.rb:166:in `instrument'
  readthis (1.2.1) lib/readthis/cache.rb:338:in `instrument'
  readthis (1.2.1) lib/readthis/cache.rb:345:in `invoke'
  readthis (1.2.1) lib/readthis/cache.rb:85:in `write'
  readthis (1.2.1) lib/readthis/cache.rb:146:in `fetch'
  lib/active_model/serializer/caching.rb:220:in `cache_check'

Dependabot can't resolve your Ruby dependency files

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::VersionConflict with message: Bundler could not find compatible versions for gem "bundler":
  In Gemfile:
    bundler (~> 1.16)

  Current Bundler version:
    bundler (2.2.15)

Your bundle requires a different version of Bundler than the one you're running.
Install the necessary version with `gem install bundler:1.17.3` and rerun bundler using `run.rb _1.17.3_`

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

ArgumentError: marshal data too short when nil is cached

According to the Cache Store Guide it is valid to cache nil, and this happens occasionally due to my use of fetch. However it seems it is attempting to marshal nil, which causes an error

Rails.cache.write('foo', nil)
# => "OK"
Rails.cache.read('foo')
# ArgumentError: marshal data too short

This has been tested in Rails 4.2.3 and 4.2.4, with Ruby 2.2.2. It works fine using other cache stores, such as memory_store

Disable caching when redis is down

I am using readthis for Rails caching with shared ElasticCache Redis instance in AWS. I'd like to be able to serve uncached content if Redis goes down (better slow site than dead one).

Any suggestions on how to do it?

Store values without serializer prefix

Perhaps this is just me nitpicking but it would be nice to store and retrieve values without any prefix. Any thoughts of making the serializer prefix optional? I feel like it should be an option to include it and not be there by default. It seems like Marshal is most commonly used by other libraries so it would be nice to switch to/from Readthis without prefixes in the values, and explicitly introduce other serializer and a prefix. Or am I missing some benefit of this current behaviour?

Consider not defaulting to hiredis since it isn't included as a dependency

Hi. This gem defaults to hiredis, but since hiredis is no longer included as a dependency, if I don't have hiredis in my Gemfile, then the connection will fail. In my opinion, it doesn't make sense to default to hiredis when it isn't included as a dependency by default. In order for readthis to work out of the box, it is required for a client app to add hiredis to their Gemfile. Therefore, why not include it as a dependency by default?

If you don't want to have hiredis as a dependency for readthis, then I think the default should be the ruby driver, and if people want to use hiredis, they can add the gem to their Gemfile and specify the driver during initialization.

Thoughts?

uninitialized constant ActiveRecord::Type from Rails.cache.fetch

Hey, I'm not fully sure if the exception I keep on getting is related to this gem; it started appearing recently, when I updated gems and migrated from dalli to redis cache and started using readthis.

And it appears for a kinda bad scenario, where there are four request coming for the cache with different id's at the same time... More below.

The setup:

  • Readthis (1.2.0)
  • Rails and it's parts (4.1.14.2)
  • Ubuntu 14.04.4 LTS
  • Phusion Passenger 5.0.26 (apache)
  • Rubygems: 2.6.2
  • Ruby: 2.2.1 (2015-02-26 patchlevel 85) [x86_64-linux]
  • Redis server v=3.0.7 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=95910942602714d9

Relevant config: code here

  config.cache_store = :readthis_store, {
        expires_in: 1.weeks.to_i, #default
        namespace: 'cache',
        redis: { url: ENV.fetch('REDIS_URL'), driver: :hiredis }
    }
    Readthis.fault_tolerant = true

So it exception comes from the line below: Full code here

@summary = Rails.cache.fetch("points_#{@course.id}_admin_#{current_user.administrator?}/", expires_in: 1.minutes) do

So, I've got four items wget -qO- https://example.com/courses/:id/points &> /dev/null in cron running every minute to keep the cache warm. Hacky, right :D.
Anyways, all of these errors are for the last url in the cronjobs. So that's basically what leads me to believe that it could be either badly configured redis or maybe this gem / rails cache related issue.
During last 24 hours, it seems, that I have gotten this exception seven times.

…pport-4.1.14.2/lib/active_support/inflector/
methods.rb: 240:in `const_get'
…pport-4.1.14.2/lib/active_support/inflector/
methods.rb: 240:in `block in constantize'
…pport-4.1.14.2/lib/active_support/inflector/
methods.rb: 236:in `each'
…pport-4.1.14.2/lib/active_support/inflector/
methods.rb: 236:in `inject'
…pport-4.1.14.2/lib/active_support/inflector/
methods.rb: 236:in `constantize'
…14.2/lib/active_support/core_ext/string/
inflections.rb:  66:in `constantize'
…upport-4.1.14.2/lib/active_support/core_ext/
marshal.rb:  10:in `rescue in load_with_autoloading'
…upport-4.1.14.2/lib/active_support/core_ext/
marshal.rb:   6:in `load_with_autoloading'
…/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
entity.rb:  67:in `load'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb:  66:in `block in read'
….2.1/gems/connection_pool-2.2.0/lib/
connection_pool.rb:  64:in `block (2 levels) in with'
….2.1/gems/connection_pool-2.2.0/lib/
connection_pool.rb:  63:in `handle_interrupt'
….2.1/gems/connection_pool-2.2.0/lib/
connection_pool.rb:  63:in `block in with'
….2.1/gems/connection_pool-2.2.0/lib/
connection_pool.rb:  60:in `handle_interrupt'
….2.1/gems/connection_pool-2.2.0/lib/
connection_pool.rb:  60:in `with'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb: 346:in `block in invoke'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb: 338:in `block in instrument'
…vesupport-4.1.14.2/lib/active_support/
notifications.rb: 161:in `instrument'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb: 338:in `instrument'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb: 345:in `invoke'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb:  63:in `read'
…s/ruby-2.2.1/gems/readthis-1.2.0/lib/readthis/
cache.rb: 142:in `fetch'
/home/tmc/srv/mooc/app/controllers/
points_controller.rb:  21:in `index'
….1.14.2/lib/action_controller/metal/
implicit_render.rb:   4:in `send_action'
…ms/actionpack-4.1.14.2/lib/abstract_controller/
base.rb: 189:in `process_action'
…pack-4.1.14.2/lib/action_controller/metal/
rendering.rb:  10:in `process_action'
…tionpack-4.1.14.2/lib/abstract_controller/
callbacks.rb:  20:in `block in process_action'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb: 113:in `call'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb: 113:in `call'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb: 552:in `block (2 levels) in compile'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb: 502:in `call'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb: 502:in `call'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb:  86:in `run_callbacks'
…tionpack-4.1.14.2/lib/abstract_controller/
callbacks.rb:  19:in `process_action'
…ionpack-4.1.14.2/lib/action_controller/metal/
rescue.rb:  29:in `process_action'
…r/rails_ext/action_controller/metal/
instrumentation.rb:  34:in `block in process_action'
…vesupport-4.1.14.2/lib/active_support/
notifications.rb: 159:in `block in instrument'
….14.2/lib/active_support/notifications/
instrumenter.rb:  20:in `instrument'
…vesupport-4.1.14.2/lib/active_support/
notifications.rb: 159:in `instrument'
…r/rails_ext/action_controller/metal/
instrumentation.rb:  21:in `process_action'
…4.1.14.2/lib/action_controller/metal/
params_wrapper.rb: 250:in `process_action'
….14.2/lib/active_record/railties/
controller_runtime.rb:  18:in `process_action'
…ms/actionpack-4.1.14.2/lib/abstract_controller/
base.rb: 136:in `process'
…/gems/actionview-4.1.14.2/lib/action_view/
rendering.rb:  30:in `process'
…ems/actionpack-4.1.14.2/lib/action_controller/
metal.rb: 196:in `dispatch'
….1.14.2/lib/action_controller/metal/
rack_delegation.rb:  13:in `dispatch'
…ems/actionpack-4.1.14.2/lib/action_controller/
metal.rb: 232:in `block in action'
…pack-4.1.14.2/lib/action_dispatch/routing/
route_set.rb:  80:in `call'
…pack-4.1.14.2/lib/action_dispatch/routing/
route_set.rb:  80:in `dispatch'
…pack-4.1.14.2/lib/action_dispatch/routing/
route_set.rb:  48:in `call'
…ionpack-4.1.14.2/lib/action_dispatch/journey/
router.rb:  73:in `block in call'
…ionpack-4.1.14.2/lib/action_dispatch/journey/
router.rb:  59:in `each'
…ionpack-4.1.14.2/lib/action_dispatch/journey/
router.rb:  59:in `call'
…pack-4.1.14.2/lib/action_dispatch/routing/
route_set.rb: 690:in `call'
…/gems/ruby-2.2.1/gems/rack-cors-0.4.0/lib/rack/
cors.rb:  80:in `call'
…l/rvm/gems/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
etag.rb:  23:in `call'
…/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
conditionalget.rb:  25:in `call'
…l/rvm/gems/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
head.rb:  11:in `call'
…1.14.2/lib/action_dispatch/middleware/
params_parser.rb:  27:in `call'
…npack-4.1.14.2/lib/action_dispatch/middleware/
flash.rb: 254:in `call'
…-2.2.1/gems/rack-1.5.5/lib/rack/session/abstract/
id.rb: 225:in `context'
…-2.2.1/gems/rack-1.5.5/lib/rack/session/abstract/
id.rb: 220:in `call'
…ack-4.1.14.2/lib/action_dispatch/middleware/
cookies.rb: 562:in `call'
…activerecord-4.1.14.2/lib/active_record/
query_cache.rb:  36:in `call'
…record/connection_adapters/abstract/
connection_pool.rb: 621:in `call'
…k-4.1.14.2/lib/action_dispatch/middleware/
callbacks.rb:  29:in `block in call'
…activesupport-4.1.14.2/lib/active_support/
callbacks.rb:  82:in `run_callbacks'
…k-4.1.14.2/lib/action_dispatch/middleware/
callbacks.rb:  27:in `call'
…k-4.1.14.2/lib/action_dispatch/middleware/
remote_ip.rb:  76:in `call'
…4.2/lib/action_dispatch/middleware/
debug_exceptions.rb:  17:in `call'
…14.2/lib/action_dispatch/middleware/
show_exceptions.rb:  30:in `call'
…-2.2.1/gems/railties-4.1.14.2/lib/rails/rack/
logger.rb:  38:in `call_app'
…-2.2.1/gems/railties-4.1.14.2/lib/rails/rack/
logger.rb:  20:in `block in call'
…esupport-4.1.14.2/lib/active_support/
tagged_logging.rb:  68:in `block in tagged'
…esupport-4.1.14.2/lib/active_support/
tagged_logging.rb:  26:in `tagged'
…esupport-4.1.14.2/lib/active_support/
tagged_logging.rb:  68:in `tagged'
…-2.2.1/gems/railties-4.1.14.2/lib/rails/rack/
logger.rb:  20:in `call'
…ms/request_store-1.3.0/lib/request_store/
middleware.rb:   9:in `call'
…-4.1.14.2/lib/action_dispatch/middleware/
request_id.rb:  21:in `call'
…/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
methodoverride.rb:  21:in `call'
…vm/gems/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
runtime.rb:  17:in `call'
…m/gems/ruby-2.2.1/gems/rack-1.5.5/lib/rack/
sendfile.rb: 112:in `call'
…/ruby-2.2.1/gems/railties-4.1.14.2/lib/rails/
engine.rb: 514:in `call'
…-2.2.1/gems/railties-4.1.14.2/lib/rails/
application.rb: 144:in `call'
…ruby-2.2.1/gems/railties-4.1.14.2/lib/rails/
railtie.rb: 194:in `public_send'
…ruby-2.2.1/gems/railties-4.1.14.2/lib/rails/
railtie.rb: 194:in `method_missing'
…lib/phusion_passenger/rack/
thread_handler_extension.rb:  97:in `process_request'
…ib/phusion_passenger/request_handler/
thread_handler.rb: 152:in `accept_and_process_next_request'
…ib/phusion_passenger/request_handler/
thread_handler.rb: 113:in `main_loop'
…c/ruby_supportlib/phusion_passenger/
request_handler.rb: 416:in `block (3 levels) in start_threads'
…-5.0.26/src/ruby_supportlib/phusion_passenger/
utils.rb: 113:in `block in create_thread_and_abort_on_exception'

Overriding expiration period

I have the following configuration in my development.rb configuration file:

config.cache_store = :readthis_store, {
    expires_in: 1.day.to_i,
    namespace: 'xxx',
    redis: { host: config.redis_host, port: config.redis_port, db: config.redis_db_for_caching, driver: :hiredis }
}

In my controller, there are pieces of data I want to cache for different time periods.

# example 1
data = Rails.cache.fetch(:data_object_one, expires_in: 15.minutes) do
    # build expensive data structure
end

# example 2
data = Rails.cache.fetch(:data_object_two, expires_in: 4.hours) do
    # build expensive data structure
end

The data is getting stored in Redis, but the TTL starts counting down from 86400 (1 day), not the specific time period I'm trying to use. Is the expires_in option not supported?

Session store

Is there any advantage for this project to have its own ActionDispatch session store over the ActionDispatch::Session::CacheStore?

If not, perhaps it should be mentioned in readme.md that it's possible to use this gem for storing sessions?

undefined method `[]' for nil:NilClass Error when cache method is called

I'm trying to use this gem and get this error when cache is called in a haml template.
While debugging it, I found that Readthis::Cache.read is called with nil as the second parameter for options.

code in haml:

- cache 'footer_links' do
  %h4
    ...

Stack trace: https://gist.github.com/salimhb/8f1b1c5a2e7c35f09dc2
My current bundle: https://gist.github.com/salimhb/f83f3ddb8b27226a626c

Am I doing anything wrong or is this a bug?

A word about delete_matched

As stated in the Readme the delete_matched is an O(n) operation. However this is what sometimes really is needed, regardless of the performance hit.
redis-activesupport gem provides this api with a warning in the documentation. After all this is providing sharp knives, isn't it?

Would you mind adding a delete_matched method anyway?

Serializers should be configurable (in initializer?)

As you moved to the binary format for flags and since the hash is frozen, no further serializers can be added anymore, e.g. Oj

The solution would be to allow user to add custom serializers (e.g. in a rails initializer file), something like

Readthis.add_serializer(Oj)

Since having a frozen hash for serializer is a very good thing once the app is configured I'd also add a method to lock them, like

Readthis.freeze_serializers!

The drawback is that user needs to call that method manually, in Rails code you might introduce a railtie to freeze that hash after app initialization.

Also if you want to have some more slots you can refactor the binary flag to use 4 bytes for serializer I used only 3 bits because no user will use more than 3 or 4 of them, but you still have 4 bits available for whatever you need. Otherwise you can shift the compressed flag to the left side using 0x80 as value for COMPRESSED_FLAG in this way the 4 unused bits will remain in the center part of the byte.

Sorry to bother you with all these requests, but you're building a very good library and I'd like to use it in my projects.

Gem is incompatible with MRI Ruby 2.0.0p247

Because of this line in the sources, it fails to run on MRI Ruby 2.0.0p247
Object.const_defined?('ActiveSupport::Notifications')

The exception is:

NameError: wrong constant name ActiveSupport::Notifications

Feature request: allow per call configuration (compression, marshal, etc)

Following #13 I was wondering if you may implement per call configuration, i.e. instead of defining marshal and compression at cache instance level allow the user to override them for the single call i.e.

Rails.cache.fetch(:some_key, compress: true, marshal: Oj) { Some.json.service }

This will likely introduce some overhead in your code but honestly I'd prefer to have a (little slower) feature complete library instead of a fastest but less configurable cache object.

The main reason to switch for me is to get rid of memcache, not to improve performances and I was looking for a feature complete library.

I tried to compare plain strings vs structure write and reading in redis and the performance impact seems to be negligible according to these benchmarks, except for hmset + expire, but we're still talking of more than 1500 calls/sec, put the network latency in the equation (almost all applications don't have redis available on localhost) and this won't matter anymore.

I did not tuned my redis installation with your suggestions for the benchmarks so doing some fine tuning might improve things.

So for 1.0 you may switch to structure saving instead of plain strings allowing to save metadata within the saved value and thus allowing per call options.

What do you think?

1.4.0 UnknownCommandError locating mexpire script.

Trying to access the cache in rails I get the following error:

Readthis::UnknownCommandError: unknown command 'mexpire'
from .../readthis-1.4.0/lib/readthis/scripts.rb:47:in `rescue in load_script!'

It seems that the gem can't find the lua file?

0.8.0 not compatible with activesupport 4.2.4?

I just updated to readthis 0.8.0 and activesupport 4.2.4 and am now getting an exception when precompiling assets for my rails app:

remote:        Running: rake assets:precompile
remote:        rake aborted!
remote:        ArgumentError: wrong number of arguments (2 for 0..1)
remote:        /tmp/build_97046e5596bfb10b8c3b0e5c337b2a9a/vendor/bundle/ruby/2.2.0/gems/readthis-0.8.0/lib/readthis/cache.rb:41:in `initialize'
remote:        /tmp/build_97046e5596bfb10b8c3b0e5c337b2a9a/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/cache.rb:60:in `new'
remote:        /tmp/build_97046e5596bfb10b8c3b0e5c337b2a9a/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/cache.rb:60:in `lookup_store'
remote:        /tmp/build_97046e5596bfb10b8c3b0e5c337b2a9a/vendor/bundle/ruby/2.2.0/gems/railties-4.2.4/lib/rails/application/bootstrap.rb:76:in `block in <module:Bootstrap>'
...

ruby 2.2, deploying to heroku. please let me know if you need any more info to reproduce.

does not work with rails5 cached collection renderer

So, Rails5 introduced multi fetching when rendering a collection in partial. It does not use fetch_multi but calls directly read_multi instead, thus treating nil values as a valid cached contents. This leads to such partials being considered as "present" in cache and just being nil

As seen here the hits are counted as a size of result returned by multi_read. Later to check if there's a hit for a particular partial it calls fetch on the result hash.

Abstract ActiveSupport::Cache::Store#read_multi treats nil as cache miss

To make readthis a drop-in replacement you'll sadly have to make a backwards-incompatible change to read_multi

Reconnect on unicorn USR2

When restarting unicorn with the USR2 signal, a new master is created and workers are forked off. In my config/unicorn.rb, before switching to readthis, I had Rails.cache.reconnect to reconnect to redis after forking. I believe this was an implementation of the redis-store gem, which you aren't using here.

How would you suggest I reconnect the unicorn worker to the redis-based cache with readthis? Thanks!

Rails 6 cache versioning error

Hi, we upgraded to Rails 6 and it's gone mostly well. While trying to push to a staging environment, the process failed with this error:

rake aborted!

You're using a cache store that doesn't support native cache versioning.
Your best option is to upgrade to a newer version of Readthis::Cache
that supports cache versioning (Readthis::Cache.supports_cache_versioning? #=> true).

Next best, switch to a different cache store that does support cache versioning:
https://guides.rubyonrails.org/caching_with_rails.html#cache-stores.

To keep using the current cache store, you can turn off cache versioning entirely:

    config.active_record.cache_versioning = false

It looks like Readthis doesn't support Rails' cache versioning. Is it possible that this functionality can be added?

Using readthis as ActiveRecord::Cache::Store with hiredis

Hi
I'd love to use redis as cache with hiredis driver, but i'm not sure how to use it here.

Would this work?

config.cache_store = :readthis_store, ENV.fetch('REDIS_URL'), {
  expires_in: 60.minutes,
  namespace: 'cache',
  driver: :hiredis
}

Dependabot can't resolve your Ruby dependency files

Dependabot can't resolve your Ruby dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Bundler::VersionConflict with message: Bundler could not find compatible versions for gem "bundler":
  In Gemfile:
    bundler (~> 1.16)

  Current Bundler version:
    bundler (2.2.15)

Your bundle requires a different version of Bundler than the one you're running.
Install the necessary version with `gem install bundler:1.17.3` and rerun bundler using `run.rb _1.17.3_`

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Getting error when Redis is down - Errno::EINVAL: Invalid argument

Hello. I am trying this out on my development environment. First thing I tested is the

Readthis.fault_tolerant = true

I am getting intermittent error using the cache when redis is down.

[275] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
Errno::EINVAL: Invalid argument
from /Users/inyigo/.rvm/gems/ruby-2.2.1@hkakabo/gems/redis-3.2.2/lib/redis/connection/hiredis.rb:17:in `connect'

[276] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
# ok here

[277] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
Errno::EINVAL: Invalid argument
from /Users/inyigo/.rvm/gems/ruby-2.2.1@hkakabo/gems/redis-3.2.2/lib/redis/connection/hiredis.rb:17:in `connect'

[278] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
# ok here

[279] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
Errno::EINVAL: Invalid argument

[378] pry(main)> Rails.cache.fetch("test_data")
Errno::EINVAL: Invalid argument

[32] pry(main)> Rails.cache.fetch("test_data")
=> nil

Works perfectly when redis is up

[304] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
[305] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
[306] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
[307] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
[308] pry(main)> Rails.cache.fetch("test_data", expires_in: 6.hours) { 'test' }
=> "test"
[367] pry(main)> Rails.cache.fetch("test_data")
=> "test"
[368] pry(main)> Rails.cache.fetch("test_data")
=> "test"
[369] pry(main)> Rails.cache.fetch("test_data")
=> "test"

Am I doing something wrong?

Usage with Puma and connection_pool

Hello!
I want to use readthis_store on multithreading Puma. I use Redis in application and for the cache. I use connection_pool
more

#initializers/redis.rb

require 'connection_pool'

REDIS = ConnectionPool.new(size: 10) { Redis.new db: 15, driver: :hiredis }



#Somewhere in the app

REDIS.with do |conn|
  conn.geoadd($DRIVER_LOCATIONS, latitude, longitude, id)
end
development.rb

config.cache_store = :readthis_store, { 
  expires_in: 1.minutes.to_i,
  namespace: 'cache',
  redis: { url: 'redis://localhost:6379/1', driver: :hiredis }
}

Should I use same connection pool for cache and if yes how can I do it?

options hash error in .8 with activesupport 4.1.12

Under readthis .8, I'm getting the error NoMethodError: undefined method `options' for #Readthis::Cache:0x000000020fea40. Reading over #11, I checked my syntax:

config.cache_store = :readthis_store, {
  expires_in: 2.weeks.to_i,
  namespace: 'cache',
  redis: { url: 'redis://localhost:6379/1', driver: :hiredis }
}

I couldnt see any problem there. So I reverted to 0.7 and tried the syntax from that point in the history:

config.cache_store = :readthis_store, 'redis://localhost:6379/1', {
  expires_in: 2.weeks.to_i,
  namespace: 'cache',
  driver: :hiredis
}

That didn't work either. So I'm assuming there's something else in my installation that is at issue. While it's not mentioned explicitly in the gemspec, I'm assuming activesupport of some version or other is required. I have 4.1.12. Are there any other requirements or other issue that might be causing this?

Commands not supported by ActiveSupport::Cache::Store

I want to expire a key and the expire command is not available in ActiveSupport::Cache::Store. In Dalli it's possible to use the Dalli client directly: https://github.com/petergoldstein/dalli/blob/master/lib/active_support/cache/dalli_store.rb#L77. Would something similar be possible with readthis? Or how about adding additional methods like expire which is what redis-activesupport (redis-store) has done https://github.com/redis-store/redis-activesupport/blob/master/lib/active_support/cache/redis_store.rb#L190

Let us pass our own Connection Pool?

Hi,

I'm really looking forward to trying this out. Looking a bit at the source code, it looks like you don't allow for me to provide my own connection pool? To maintain predictable redis connection counts, I want to share my Sidekiq client pool with my session store and maybe my IdentityCache Pool as well. Would appreciate any insights here if you've benchmarked this. My plan:

  • cache_store: Keep memcached for now. Cheaper for larger blobs (views)
  • identity_cache_store: readthis (currently use memcached)
  • session_store: readthis (currently use redis-store)

Thanks.

Logging?

Hi there.

I'm switching to readthis from redis_store, and am seeing unexpected cache keys being written. In trying to debug this, I was looking for logging of some sort: Observing which keys and values are being written or read per operation. Am I overlooking existing functionality, or might this be something you'd support?

Cheers.

Incrementing and decrementing key does not preserve its expiration (i.e. TTL)

Ruby Version: 2.3.5
Readthis Version: 2.0.2
Environment: macOS Sierra 10.12.6

If you open up a Redis command line, set a key/value with a TTL (time to live), then increment the key, the TTL stays:

redis> SET x 1 'EX' 99999
OK
redis> TTL x
(integer) 99997
redis> INCR x
(integer) 2
redis> TTL x
(integer) 99994

If you do the same with a Readthis client however, the TTL will disappear:

irb(main):006:0> readthis_client.write('x', '1', expires_in: 99999)
=> "OK"
irb(main):007:0> readthis_client.pool { |c| c.ttl('x') }
=> 99997
irb(main):008:0> readthis_client.increment('x')
=> 2
irb(main):009:0> readthis_client.pool { |c| c.ttl('x') }
=> -1

It seems the cause is #increment and #decrement don't call the incr and decr Redis commands, but instead have their own implementation where they read the existing value and then write over it. But while doing so, these methods don't preserve the TTL of the key.

It seems we're not calling incr and decr because Redis can't actually increment or decrement values for Readthis if Readthis encodes and marshals all values it sends to Redis. See 139553b

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.