Comments (17)
Both sound like good ideas. Look for those additions in the near future.
from agoo.
Hi there!
I came across from the web page benchmark page (congrats with being the fastest in ruby). Running the Rails app in clustered mode would require to have at least before_fork
hook as normally this is the place to close any connections that would be corrupted in the parent on child process exit. Like that
before_fork do
Sequel::DATABASES.each(&:disconnect)
end
Also it would be nice to have some kind of an after_worker_fork
callback to have the opportunity to do some work in the context of new spawned worker
from agoo.
I'll look into it this weekend. My unit tests worked last time I check so with any luck there is some configuration addition that is needed.
from agoo.
It worked for me on dedicated hardware, my laptop. Does ./bin/agoo -p 8080 -t 8
work on on one of you machines when not using docker?
A couple of points though. You will probably find that making the thread count too high actually decreases performance. The bottleneck on a single process is the Ruby global lock. Having more than somewhere around 8 just mean more context switching and threads tied up waiting. I usually do some benchmarking with an app and try to find the optimum number. Then increase the worker count.
from agoo.
Our app is IO bound so more threads mean we can do processing while waiting for IO to complete
I'm thinking the issue might be related to database connections and forking - I think puma forks before rails is loaded unless preload_app!
is called and we're not calling that as it was too challenging to coordinate various gems initialisation
Which basically means we're sacrificing memory savings that can be gained with forking
from agoo.
IO bound outside Ruby I take it. Makes sense.
If the database connection is being made before forking I could see how that could be a problem. Agoo is independent of rails so I'm not sure when it gets initialized relative to Agoo. Depends on how it is started I suppose. I would be interested in exploring options that would make it work for you if you want to pursue this. I think the next step might be instrumenting the code with some print statements to figure out when database connections are made and when forking occurs. See if your theory is correct and after that figure out what can be added to Agoo to allow the database connections to be made after forking.
from agoo.
A fork_wrap option was added in the develop branch. Please give it a try.
from agoo.
No response, closing.
from agoo.
Hi @ohler55!
I was wondering if it might be possible to differentiate between when a worker is forked vs. when a worker is booted - I'm trying to migrate from Puma to Agoo, and in my containerized Rails environment, we have the following:
before_fork do
ActiveRecord::Base.connection_pool.disconnect! if defined?(ActiveRecord)
end
on_worker_boot do
ActiveRecord::Base.establish_connection if defined?(ActiveRecord)
end
Do you have any examples on how that can be migrated to use the new fork_wrap
option?
Thanks!
from agoo.
What does it mean to have a worker booted versus the app being forked?
from agoo.
Based on the puma documentation:
before_fork
- If you are preloading your application and using Active Record, it's recommended that you close any connections to the database before workers are forked to prevent connection leakage.
on_worker_boot
- The code in theon_worker_boot
will be called if you are using clustered mode by specifying a number ofworkers
. After each worker process is booted, this block will be run. If you are using thepreload_app!
option, you will want to use this block to reconnect to any threads or connections that may have been created at application boot, as Ruby cannot share connections between processes.
So both options seem to account for the possibility of the app being preloaded when running in clustered mode, where the app boot and code loading happens before the workers are forked, as a means to reduce memory footprint - however without setting the on_worker_boot
, leaves the possibility of threads not being connected to the database yet
from agoo.
Since we wouldn't be preloading the app with agoo, I guess the on_worker_boot
doesn't matter - we would only need to ensure that ActiveRecord is disconnected before_fork, and that it's connected after_fork between the master and worker processes: https://devcenter.heroku.com/articles/concurrency-and-database-connections#multi-process-servers - can that be achieved in the fork_wrap
option?
from agoo.
If a forker object is set the before method will be called before forking and the after will be called after forking. I think that should work for you, yes?
from agoo.
@ohler55 that would be perfect!
from agoo.
Right, that is the current behavior.
from agoo.
Where might the before block be defined for agoo's config if running as part of Rails through rackup?
from agoo.
Look at example/quux.rb.
from agoo.
Related Issues (20)
- Fragment usage HOT 17
- CORS issues HOT 8
- Agoo dumps core on malformed SDL HOT 5
- Agoo Fragment Cycle Security Vulnerability HOT 2
- Disabling introspection for GraphQL HOT 12
- Changes on args or req does not get fowarded to the other calls HOT 9
- Static asset with space in filename HOT 4
- gem install agoo fails on MacOS Ventura 13.2.1 HOT 9
- Static .txt files are not served with utf-8 encoding HOT 8
- Agoo crash and exit HOT 7
- CLI options for SSL/TLS HOT 3
- localhost is crashing as bind address HOT 4
- 502 via nginx proxy_pass HOT 3
- TypeError: no implicit conversion of Array into String HOT 4
- rackup -r agoo -s agoo Doesn't Work HOT 8
- Out of curiousity would anything have been lost or gained by using ABI, or FFI, or *.SO over the current API method of directly calling in the Ruby C file headers? HOT 3
- Configuring worker count and thread count when used with Rails HOT 5
- Port cleanup on shutdown? HOT 3
- REMOTE_ADDR not set on ENV HOT 13
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from agoo.