fetlife / rollout Goto Github PK
View Code? Open in Web Editor NEWFeature flippers.
License: MIT License
Feature flippers.
License: MIT License
I noticed that my requests were slow when looking up whether a user should see a feature, especially as I introduced more features with more user ids. I was interested in improving the .active?(feature, id)
performance, which is why I ran into this.
When working with a feature with a large number of user ids, feature retrieval takes a while (130ms
for a feature with 100k
user ids).
I narrowed it down to the .to_set
call when creating the feature:
https://github.com/fetlife/rollout/blob/master/lib/rollout.rb#L18
Using the set improves lookup performance, but costs a lot up front. The benchmarks in this PR didn't take into account the cost to create the set.
During a response/request cycle, it's likely that we create the feature once, and do a couple operations on it (e.g. .active?
), so optimizing the lookup performance might not be appropriate.
My recommendation would be to make using sets optional. There may be some cases where you have a long living instance of a Feature
and want to do many lookups, but for the most part, I think using an array is an OK compromise.
Here's a benchmark that simulates the creation of the list of users from a raw string of comma-delimited user ids and checks whether a user id is in the list (set vs array vs string).
# create a set of raw_users in a string
raw_users = []
100_000.times {|num| raw_users << num }
raw_users = raw_users.join(',')
iterations = 100
# benchmark the way that rollout does an `active?` check w/ a set
## 10.880000 0.180000 11.060000 ( 11.090479) -> each operation takes about 110ms
Benchmark.bm do |bm|
# joining an array of strings
bm.report do
iterations.times do |i|
users = (raw_users || "").split(",").map(&:to_s).to_set
users.include?(i)
end
end
end
# benchmark checking for a value in an array
## 2.430000 0.080000 2.510000 ( 2.520499) -> each operation takes about 25ms
Benchmark.bm do |bm|
# joining an array of strings
bm.report do
iterations.times do |i|
users = (raw_users || "").split(",")
users.include?(i)
end
end
end
# benchmark checking for a value in the string
## 0.000000 0.000000 0.000000 ( 0.005260) -> each operation takes about .05ms
Benchmark.bm do |bm|
# joining an array of strings
bm.report do
iterations.times do |i|
users = (raw_users || "")
!!users.match(/^#{i},|,#{i},|,#{i}$/)
end
end
end
I don't know the proper place to file this,
but the chromium team confirms it is a FetLife bug.
https://bugs.chromium.org/p/chromium/issues/detail?id=884508#c3
It should be possible to store additional description of feature so you can get more about the feature.
Hi,
The User Percentages rollout section, defines the algorithm as using CRC32, which seem to expect only integers to determine percentage rollouts. How will this feature interact with the knowledge that our User's primary key is not an auto incrementing integer, but a UUID generated by Postgres using uuid_generate_v4()
. Should I just avoid this feature altogether? Will having UUID's as my user's primary key break anything else in this gem?
Just stumbled across the gem, and it looks interesting, I'm wondering how receptive you would be to pull requests that include support for other backends (keeping the default redis).
Figured I'd start the conversation before spending a weekend in code :)
Rollout seems to get confused when restarting a unicorn setup, because it's using the old Redis connections.
ActionView::Template::Error (Tried to use a connection from a child process without reconnecting. You need to reconnect to Redis after forking.):
Is there any workaround currently? The way I see it, there's no straightforward way to just reinitialize an existing Rollout with a new Redis object, and I'd rather not put the entire Rollout initializer in the unicorn after_fork
block.
it would be great to get a list of flags currently available to a user..
When Redis's AOF persistence is enabled, activating x specific users causes quadratic writes to disk, since each additional user causes the entire set of activated users to be written.
Considering only the IDs that get written to the AOF, and assuming 8 bytes per ID, we have:
Number of users added | Bytes written to AOF |
---|---|
x | 4 * x ** 2 + 4 * x bytes |
1 | 8 bytes |
1_000 | 3.8 megabytes |
10_000 | 381 megabytes |
100_000 | 37.3 gigabytes |
(For some background, I wanted to enable a feature for tens of thousands of users, selected based on a costly-to-compute criterion.)
When seeing the list of events available from a group or a personal profile in my experience has been not that simple. Having to scroll through a dozen past events to find any dates of that same day or future days is not user-friendly.
I see the merit of having access to past events; however, being able to identify events of the current day or future days is crucial for proper calendar implementation.
I suggest implementing a two-tier system: a repository of past events to attended or considered events and having a list of current and upcoming events as is currently implemented and available.
The method get
should return nil in case rollout is no exist.
This way the behaviour of the method is more predictable.
What do you think?
Now that user ids are hashed before checking against percentage, you can more "randomly" distribute which users get which features early.
While I could be wrong, currently, all features with a rollout percentage of 10% will be available to the same 10% of users. This doesn't adequately test all different combinations of released features. It also means that the same users will always be the first to gain access to a new feature.
If the hash is partly based on the feature name, there should some users who don't get feature A or B, some with just A, some just B and some with both.
I've been making a UI specific to my company, and I feel like that it would be just easier if I could pass IDs to activate_user instead of passing the id to the controller action, looking up the record, and THEN passing that.
Seems like using IDs instead of user objects is not supported when using groups.
E.g:
Define a group for all users which email ends with @acme.com:
rollout.define_group(:acme) do |user|
user.email.end_with?('@acme.com')
end
Activate the chat feature and add "acme" group:
rollout.activate_group(:chat, :acme)
Enable the "chat" feature only to the 0.1% of those users:
rollout.activate_percentage(:chat, 0.1)
See if the chat feature is active for ID "42":
rollout.active?(:chat, "42")
Raises NoMethodError: undefined method 'email' for "42":String
I just released redis-rb 4.0.0 RC1.
Can you please test against it and report back?
Thank you!
The README suggests that you assign the Rollout.new
instance to the global variable $rollout
. This is not generally recommended for Rails applications, discouraged by Rubocop, and mostly because of thread safety and for a few other reasons, if I remember correctly. Why not suggest something like this for a Rails application?
# config/initializers/rollout.rb
Feature = Rollout.new(Redis.current).freeze
# ...
# somewhere else
Feature.active? :chat
Just throwing this idea out there, and if it would be supported by this gem:
Instead of per-user rollouts; per-model rollouts as well.
Some application might have features that if rolled out, should only be rolled out to anything interacting with a particular model.
e.g. All new repositories should have this feature enabled; but all old repositories shouldn't
Would it make sense to roll this into this gem? I am willing to spend the time doing it, just want to get some input first! (e.g. what type of interface, etc).
It might actually might make sense to drop groups or a new data format for this feature as well; cause groups could be a Value type model (that is used in the namespace); and .active?('feature', RolloutGroup('foo')).
Thoughts? Input? Discussion?
version : 2.0.0
Setting a feature on for all :
$rollout.activate_group(:one_feature, :all)
> "OK"
Checking if the feature is on globally :
$rollout.active?(:one_feature)
> false
Checking if the feature is on for one account :
$rollout.active?(:one_feature, @account)
> true
All accounts do return true still but not the global check.
I have tried to limit users by 20 percent but seemed all the users can see the view that I blocked
- if $rollout.active? :percentage_rollout, current_user
%h2
Hello
=current_user.first_name
my initial instance
$name_space = Redis::Namespace.new(Rails.env, redis: $redis)
$rollout = Rollout.new($name_space)
$rollout.define_group(:percetage_users) do |user|
user
end
$rollout.activate_users(:percentage_rollout, User.all)
$rollout.deactivate_group(:percentage_rollout, :percentage_users)
$rollout.activate_percentage :percentage_rollout, 20
please correct me if what i did wrong here?, thanks
The main website has email notifications and a calendar sprite but what I feel should be the next step with events should be the calendar integration so you don't have to go to the site and check or manually input the event into a separate calendar.
Thoughts?
@jamesgolick, do you have a suggestion re: how to use rollout when concerning yourself with thread-safety? The default suggestion for using a global sort of concerns me as far as threads (i.e. non-MRI) go, but it seems based on a cursory read through the README that that's in no way a requirement. Have you had to deal with rollout in a threading environment?
You wrote, "As of rollout-2.x, only one key is used per feature for performance reasons. The data format is percentage|user_id,user_id,...|group,_group.... This has the effect of making concurrent feature modifications unsafe, but in practice, I doubt this will actually be a problem.
This also has the effect of rollout no longer being dependent on redis. Just give it something that responds to set(key,value) and get(key)."
Then what is used to save the settings?
Which is preferred for a production live system? With Redis or without?
Thank you.
If I put rollout in my Gemfile, I get version 1.2.0. I see 2.0.0 has been out for some time now but I assume Bundler will not pull it because there's a letter at the end of the version number? James, is this intentional? Is 2.0.0x still in beta?
There's also an old and confusing VERSION file in the root of the gem - can that be removed?
It is my understanding that json is not part of ruby standard library, hence needs to be declared as a dependency of rollout as of 7b7bfc5.
Is it possible to remove all references from a rollout flag so they no longer show up in our rollout_ui? There are no delete or remove methods that are accessible.
Hi,
I've read the source code and tried a few feature toggles in one of my applications.
If I understand correctly, a feature is disabled by default. Am I right ?
If so, is there a good way to have it enabled by default ?
Thanks
Have y'all thought about adding account support?
Our system has the concept of accounts which are collections of users. We would like to enable a feature for a account without having to add each individual user.
I found the feature setting is retrieved directly from redis often if used on every request. Should we memoize it?
Right now there is no obvious way to fetch all features that were stored in the Redis database using Rollout's DSL.
Here are couple of examples why it is hard for rollout dependencies:
https://github.com/jrallison/rollout_ui/blob/master/lib/rollout_ui/monkey_patch.rb#L5
https://github.com/nfedyashev/rollout-js/blob/master/lib/rollout_js/redis_middleware.rb
@jamesgolick Do you think this feature should be added to the Rollout so that others developers can rely on it and not reinvent this finder?
The global variable and global redis connection would seem to cause problems in multithreaded servers, which is becoming a lot more common in Rails deployments (puma is now the recommended server by heroku for Rails). I would love to see rollout handle this seamlessly.
Inspired by how the django equivalents nexus and gargoyle integrate:
https://github.com/disqus/gargoyle
I was just curious, has some thought / work been put into this already, or is there a different "admin thing" that would be preferable? Obviously, I've made a lot of assumptions here, such as rails. Possibly better suited as a separate project.
When I have completed beta development for a feature and no longer need to reference the feature in Redis from my application, I don't want that entry to hang around in Redis forever. I can easily delete the entry from Redis CLI, but it seems like there should be a public interface to delete the feature from Redis $rollout.delete(:old_feature_rolled_out_years_ago)
.
Is there a reason this was omitted?
Bump version so #23 gets published.
Hi all,
I just wanted to inform all the people that care about rollout gem that we @ Fiverr built a new client app for managing our rollouts.
https://github.com/fiverr/rollout_dashboard/
The algorithm implemented in user_in_percentage?
is flawed, because it doesn't distribute evenly.
(1..1_000).select {|x| Zlib.crc32(x.to_s) % 100 < 25 }.count # => 271. Expected value would be 250
As a side note, CRC32 is prone to hash collisions (see here).
This method simply replace the users of a given feature with a given array.
has anyone seen this happen? We've started to see some flags change from 0 to 100 (ie being enabled), seemingly randomly, eg:
irb(main):266:0> namespace.get "feature:my_flag"
=> "0||"
irb(main):267:0> namespace.get "feature:my_flag"
=> "100|1353861,1353862,1348715,1353863|"
Time between these 2 commands was maybe 20mins, the code base doesn't touch flag permissions, directly, we just set them via the console.. any thoughts?
Is there a way to check what % of users have a given feature flag?
I haven't been able to figure out how to do that.
We need to release a new version of this gem to catch it up to the source. It's outdated.
RubyGems says rollout is now up to 2.5.0
. Can you add a tag and release for this version here in GitHub?
To reproduce: define a group with a | character in its name, and activate the group for a feature that has some additional data. Next, check whether the feature is active. You will likely run into a JSON parsing exception.
The offending line:
rollout/lib/rollout/feature.rb
Line 14 in 0b07edf
The | character in the group name causes garbage data to be assigned to the raw_data variable, which is subsequently parsed as JSON on line 18, resulting in the exception.
How to fix: I guess the group names should be serialized somehow. You could also disallow the | character in group names, I guess.
Something like this?
class Rollout
# ...
+ def to_hash
+ results = {}
+ features.each do |feature|
+ with_feature(feature) do |f|
+ results[feature] = f.to_hash
+ end
+ end
+ results
+ end
end
Maybe I'm using it wrong, or its mentioned somewhere in the docs, and I have not read it properly.
But if you do this :
user = User.first
$rollout.activate_group(:chat, :all) # You'd assume this activates feature for everyone, and it does.
$rollout.deactivate_user(:chat, user) # I'm assuming this should only deactivate the feature for this user, everyone else should still have it active.
$rollout.active?(:chat, user) # This should return false, but returns true.
I am looking at updating rollout dependency in our project. We are at 2.3.0. There is no 2.3.0 tag in this repo hence getting a diff between 2.3.0 and 2.4.1 is not possible. 2.4.1 tag also does not exist.
After upgrading to version 4.2.0 of the redis
gem, I see these deprecation warnings:
`Redis#exists(key)` will return an Integer in redis-rb 4.3, if you want to keep the old behavior, use `exists?` instead.
To opt-in to the new behavior now you can set Redis.exists_returns_integer = true.
(rollout-2.4.5/lib/rollout.rb:299:in `exists?โ)
Seems like rollout should use exists?
for redis 4.2.0 and higher.
I needed to check if there were any rollouts active for a specific user. Could be a useful feature.
Hello @fetlife , I'm curious as to whether this project is still being maintained? It's absolutely fine if it's not, I'd just like to know before I start using it. Thanks in advance
I know there could be multiple kinds of activations at the same time, like:
just wondering if it's possible to get those
like..
$rollout.those_with_access_to(:chat)
>
{
percent: 20,
individuals: [1,3,4]
groups: [:admin, :me]
}
idk
Is it safe to upgrade from 2.4.x to 2.5? Looking to use the Rollout UI.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.