afeld / magickly Goto Github PK

View Code? Open in Web Editor NEW

180.0 180.0 35.0 584 KB

image manipulation as a (plugin-able) service

Home Page: http://magickly.afeld.me

License: MIT License

Ruby 88.33% CSS 5.06% HTML 6.61%

magickly's People

Contributors

Stargazers

Watchers

magickly's Issues

use different ImageMagick library?

Met @maxim the other day after giving a talk about magickly, and he suggested using Skeptic as the ImageMagick (which I'll call "IM") wrapper.

I haven't been actively working on magickly in a while, and haven't been following progress on IM Ruby libraries. The issue with Dragonfly 0.x (not sure about 1.x) is that each transformation is applied in a separate convert command

Use the single-convert changes done by @tolsen forever ago
- Would want to do some cleanup/documentation before merging in – see #16
- Ideally this logic would be at the IM wrapper level, keeping magickly simple
Use RMagick
- All in memory (not sure if this is a good or bad thing)
- Read veeery mixed reviews about it when first building magickly and deciding what to use (e.g. memory leaks), though this may have improved since
Use Skeptic
- From a quick glance, seems well-thought-out, and well-suited to keeping magickly easily extensible.
- The DSL could make the existing shortcuts a lot more readable.
- Doesn't do argument format enforcement – ideally this would be outside magickly as well
- Only runs a single convert command (correct @maxim?), so theoretically should have the same performance characteristics as single-convert

Feedback/ideas welcome!

P.S. This change would obviate #48.

Timeouts when using magickly on local images

I've mounted magickly in a Padrino app (basically Sinatra) using the following in config.ru

#!/usr/bin/env rackup
# encoding: utf-8

# This file can be used to start Padrino,
# just execute it from the command line.

require File.expand_path("../config/boot.rb", __FILE__)

require 'rack/cache'

use Rack::Cache

map '/magickly' do
  run Magickly::App
end

map '/' do
  run Padrino.application
end

I can run this locally using bundle exec thin start and it works fine.

The problem is when I try and use magickly on an image that's hosted locally, like so;

http://magickly.dev:3000/magickly/?src=/images/logo.png
http://magickly.dev:3000/magickly/?src=http://0.0.0.0:3000/images/logo.png

Neither of these work and they fail with Timeout::Error

Any ideas?

Notify Airbrake, if present

...or allow configuration?

move in JS client library

We've developed something somewhat custom at @Jux, but it wouldn't be hard to abstract it a bit and add it here.

Add thin to Gemfile (but not gemspec)?

That way the cedar branch can be merged to master for deploying magickly to Heroku or somewhere directly, without requiring thin when used as a gem. On the other hand, thin isn't the only option of server to use, so maybe we don't want to force that requirement?

@jcn

Rack Protection 1.5.1 Breaks Magickly

If you are using magickly as a gem in your app and try to update it, rack protection might update and break it. You'll need to lock into 1.5.0 in your app's Gemfile.

experiment with lens flare effect

example: https://jux.com/quarks/213322

config option to restrict hostname

will make people feel safer about deploying it for production apps

make tilt_shift aspect-ratio-aware

Add support for custom filters

Automatically kill long-running converts

Not sure if it needs to be a separate job or what...

Configurable timeout

switch to ERB

no need to keep an extra dependency on HAML

switch to multi_json gem

https://github.com/intridea/multi_json

Performance stats

Hey,

Just thought I'd share some test results with you quick.

Server is an instance on AWS
Source image is huge (10800 × 5400, 14,5 MB) http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg

In case you get bored to death before you get to the bottom of this, somewhat useless piece of information, thanks.

So, I'm here...

... the server is here...

...and the src is here:

Using Siege I started a siege doing a 'resize=640' and then a few minutes later asked someone else to hit a new one, 'resize=1080', and then a few minutes later again, another person using 'resize=720'

I took 2 screenshots and a top -c output copy/paste from the server. Once as the second siege got started and then again while all 3 were running.

I was running 4 Unicorn workers using the code from my release brach here: https://github.com/eksoverzero/magickly/tree/release
There does seem to be an issue with S3 or something. I have no idea why, between an EC2 instance and a S3 bucket it would be so slow.
The 'Longest transaction' times aren't bad considering the image it's processing?

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=640


Transactions:              13209 hits
Availability:              99.91 %
Elapsed time:             867.17 secs
Data transferred:         635.99 MB
Response time:              1.49 secs
Transaction rate:          15.23 trans/sec
Throughput:             0.73 MB/sec
Concurrency:               22.76
Successful transactions:       13209
Failed transactions:              12
Longest transaction:           29.20
Shortest transaction:           0.91

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=1080

Transactions:              11821 hits
Availability:              99.90 %
Elapsed time:             902.52 secs
Data transferred:        1567.70 MB
Response time:              2.53 secs
Transaction rate:          13.10 trans/sec
Throughput:             1.74 MB/sec
Concurrency:               33.12
Successful transactions:       11821
Failed transactions:              12
Longest transaction:           30.21
Shortest transaction:           2.00

siege -d10 -c100 http://**********.co.za/\?src\=http://upload.wikimedia.org/wikipedia/commons/3/3d/LARGE_elevation.jpg\&resize\=720

Transactions:               7272 hits
Availability:              99.85 %
Elapsed time:             495.33 secs
Data transferred:         438.29 MB
Response time:              1.66 secs
Transaction rate:          14.68 trans/sec
Throughput:             0.88 MB/sec
Concurrency:               24.31
Successful transactions:        7272
Failed transactions:              11
Longest transaction:           29.15
Shortest transaction:           1.13

Here are the app and monitoring stats:

top - 14:10:33 up 10 days,  1:04,  1 user,  load average: 0.04, 0.24, 0.20
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.9%us,  1.5%sy,  0.0%ni, 94.9%id,  0.0%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:   7629456k total,  5436484k used,  2192972k free,   210020k buffers
Swap:        0k total,        0k used,        0k free,  4415104k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                         
 7810 *******   20   0  395m  96m 4176 S    2  1.3   0:13.78 unicorn worker[1] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8203 www-data  20   0 64132 3232 1020 S    2  0.0   0:03.08 nginx: worker process                                                                                                                                                            
 7813 *******   20   0  396m  97m 4176 S    2  1.3   0:13.49 unicorn worker[2] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 7816 *******   20   0  407m  94m 4184 S    2  1.3   0:13.75 unicorn worker[3] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 7807 *******   20   0  396m  97m 4176 S    1  1.3   0:14.04 unicorn worker[0] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 5275 mongodb   20   0 6781m  85m  17m S    1  1.1  24:16.05 /usr/bin/mongod --config /etc/mongodb.conf

top - 14:19:57 up 10 days,  1:14,  1 user,  load average: 0.49, 0.39, 0.32
Tasks:  81 total,   1 running,  80 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.2%us,  2.7%sy,  0.0%ni, 92.8%id,  0.0%wa,  0.0%hi,  1.2%si,  0.2%st
Mem:   7629456k total,  5314188k used,  2315268k free,   210036k buffers
Swap:        0k total,        0k used,        0k free,  4425396k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                         
10884 *******   20   0  289m  64m 3968 S    3  0.9   0:02.08 unicorn worker[2] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8204 www-data  20   0 64296 3304  932 S    3  0.0   0:06.52 nginx: worker process                                                                                                                                                            
10878 *******   20   0  289m  64m 3968 S    2  0.9   0:02.61 unicorn worker[0] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
10860 *******   20   0  289m  64m 3968 S    1  0.9   0:02.92 unicorn worker[3] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
10846 *******   20   0  289m  64m 3988 S    1  0.9   0:04.05 unicorn worker[1] -p 5000 -c ./config/unicorn.rb                                                                                                                                 
 8203 www-data  20   0 63904 3004 1024 S    1  0.0   0:07.71 nginx: worker process

Get single-convert into master

The performance gain in the single-convert branch is huge, but the factory specifications are still too complex. I believe this can be pushed down a layer to wrap Dragonfly, or moved into Dragonfly itself, or maybe it just makes sense to move off Dragonfly.

Add Rack::Cache support

Since we're now ready for cedar deployment, it probably makes sense to also include Rack::Cache. I have a branch that just uses this in all environments and uses a filesystem-based store which works quite nicely. I'm not sure of the implications of this (whether we want to give people the option to not use Rack::Cache, etc).

One thing I was thinking was to just allow it to be turned on and off using an environment variable, but I'm wondering if it just makes sense to leave it in and say "this is the way it's done."

getting 404s for some images

e.g. https://s3.amazonaws.com/craj_biznesses_pics/4a3aa8380f4452af4d6e56e848dd083162838453Screen%2BShot%2B2012-11-29%2Bat%2B3.07.02%2BAM.png

response is fine in browser - thought it might be a UA-string issue but HEAD request is fine in cURL...

Website Down

I noticed the Heroku app has gone and the examples on the README don't work. Is this easily fixable?

Drop 1.8.7 support

It is time. People can always use an earlier version of the gem/repo if they need. This could just mean removing it from the Travis build, but might try running a script to convert to 1.9-style hash syntax.

Permissions set incorrectly for lomo_mask.png

Should be 644...

Production query

Hi there,

At my organization we’re deploying a metatdata aggregation app, currently in a demo phase. The app calls on http://magickly.afeld.me/ to process thumbnails for display. We’re working on deploying our own instance of magickly for this, but aren’t quite there yet. The readme makes a reference to using the magickly demo in production . . . is this available/advisable?

Thanks!

protect_from_dos_attacks does not protect against DoS

It seems like Dragonfly's protect_from_dos_attacks doesn't take effect under Magickly.

Magickly.dragonfly.configure do |c|
  c.protect_from_dos_attacks = true
  c.secret = 'swordfish'
end

See http://markevans.github.com/dragonfly/file.URLs.html for details.

Add note to README re: master branch

We've been putting new features on the master branch, which isn't the greatest because the documentation for the gem will appear incorrectly. At minimum, add a note to the README saying some features may not be released yet, and refer them to the gem rdoc:

http://rubydoc.info/gems/magickly/frames

upgrade to Dragonfly 1.x

Resolve ActiveSupport version w/ old rubies

The gemspec does't set a max version for ActiveSupport, so the build is trying to install 4.x, which requires Ruby >= 1.9.3. Two possible solutions:

Set a max version for ActiveSupport, e.g. s.add_dependency('activesupport', '>= 2.0.0', '< 4')
Drop Ruby 1.8.7 and 1.9.2 support.

Regards,

Regis A. Despres

afeld / magickly Goto Github PK

magickly's People

Contributors

Stargazers

Watchers

Forkers

magickly's Issues

Recommend Projects

Recommend Topics

Recommend Org