madebyhiro / codem-schedule Goto Github PK

When a job is created, no state change for 'Scheduled' is inserted in the database.

A group of jobs with the same input file and different presets should be able to be added as a batch in one operation.

Certain jobs will stay in state "Scheduled" even though the schedule API endpoint is being called and there are free slots in the transcoders. When listing the details of the job it will be in state "Unknown"(?).

I have a barely-working knowledge of ruby but doesnt net::http::post_form only work with plain http?

Will open a pull request that replaces that with HTTParty::post, which I used to solve this.

I can not build the scheduler. With the current configuration.
Only if you delete the twitter "bootstrap_and_overrides.css.less" the build runs fine but without GUI.

sudo rake codem:install --trace
** Invoke codem:install (first_time)
** Invoke codem:install:install (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute codem:install:install
Creating database and running migrations....
** Invoke codem:install:install_db (first_time)
** Invoke db:create (first_time)
** Invoke db:load_config (first_time)
** Execute db:load_config
** Invoke rails_env (first_time)
** Execute rails_env
** Execute db:create
db/development.sqlite3 already exists
db/test.sqlite3 already exists
** Invoke db:migrate (first_time)
** Invoke environment
** Invoke db:load_config
** Execute db:migrate
** Invoke db:_dump (first_time)
** Execute db:_dump
** Invoke db:schema:dump (first_time)
** Invoke environment
** Invoke db:load_config
** Execute db:schema:dump
** Execute codem:install:install_db
Created codem database and ran migrations.

** Invoke codem:install:setup_transcoder (first_time)
** Invoke environment
** Execute codem:install:setup_transcoder
If you already installed one or more Codem Transcoder instances, we can add it to the Scheduler for you.

Enter the url to the Codem Transcoder instance, or press to skip:

** Invoke codem:install:configure_mailer (first_time)
** Execute codem:install:configure_mailer
To send email notifications, the Codem mailer needs to be set up.
You can do this manually in config/initializers/action_mailer.rb
Do you want to setup the mailer now? (y/n)
n

** Invoke assets:precompile (first_time)
** Execute assets:precompile
/usr/bin/ruby1.9.1 /usr/local/bin/rake assets:precompile:all RAILS_ENV=development RAILS_GROUPS=assets --trace
** Invoke assets:precompile:all (first_time)
** Execute assets:precompile:all
** Invoke assets:precompile:primary (first_time)
** Invoke assets:environment (first_time)
** Execute assets:environment
** Invoke environment (first_time)
** Execute environment
** Invoke tmp:cache:clear (first_time)
** Execute tmp:cache:clear
** Execute assets:precompile:primary
rake aborted!
cannot load such file -- less
(in /home/ubuntu-scheduler/Arbeitsfläche/codem-schedule/app/assets/stylesheets/bootstrap_and_overrides.css.less)
/var/lib/gems/1.9.1/gems/activesupport-3.2.9/lib/active_support/dependencies.rb:251:in require' /var/lib/gems/1.9.1/gems/activesupport-3.2.9/lib/active_support/dependencies.rb:251:inblock in require'
/var/lib/gems/1.9.1/gems/activesupport-3.2.9/lib/active_support/dependencies.rb:236:in load_dependency' /var/lib/gems/1.9.1/gems/activesupport-3.2.9/lib/active_support/dependencies.rb:251:inrequire'
/var/lib/gems/1.9.1/gems/tilt-1.3.3/lib/tilt/template.rb:108:in require_template_library' /var/lib/gems/1.9.1/gems/tilt-1.3.3/lib/tilt/css.rb:55:ininitialize_engine'
/var/lib/gems/1.9.1/gems/tilt-1.3.3/lib/tilt/template.rb:55:in initialize' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/context.rb:192:innew'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/context.rb:192:in block in evaluate' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/context.rb:190:ineach'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/context.rb:190:in evaluate' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:12:ininitialize'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:249:in new' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:249:inblock in build_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:270:in circular_call_protection' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:248:inbuild_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:93:in block in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/caching.rb:19:incache_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:92:in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:169:infind_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:60:in find_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:111:inblock in resolve_dependencies'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:105:in each' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:105:inresolve_dependencies'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:97:in build_required_assets' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/processed_asset.rb:16:ininitialize'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:249:in new' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:249:inblock in build_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:270:in circular_call_protection' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:248:inbuild_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:93:in block in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/caching.rb:19:incache_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:92:in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:169:infind_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:60:in find_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/bundled_asset.rb:16:ininitialize'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:252:in new' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:252:inbuild_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:93:in block in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/caching.rb:19:incache_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:92:in build_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:169:infind_asset'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/index.rb:60:in find_asset' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/environment.rb:78:infind_asset'
/var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/static_compiler.rb:19:in block in compile' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:219:inblock in each_logical_path'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:206:in block (2 levels) in each_file' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:196:ineach'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:196:in each_entry' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:204:inblock in each_file'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:203:in each' /var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:203:ineach_file'
/var/lib/gems/1.9.1/gems/sprockets-2.2.2/lib/sprockets/base.rb:217:in each_logical_path' /var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/static_compiler.rb:18:incompile'
/var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:56:in internal_precompile' /var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:70:inblock (3 levels) in <top (required)>'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:in call' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:inblock in execute'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:in each' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:inexecute'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:166:in block in invoke_with_call_chain' /usr/lib/ruby/1.9.1/monitor.rb:211:inmon_synchronize'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:159:in invoke_with_call_chain' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:152:ininvoke'
/var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:60:in block (3 levels) in <top (required)>' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:incall'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:in block in execute' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:ineach'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:in execute' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:166:inblock in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:211:in mon_synchronize' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:159:ininvoke_with_call_chain'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:152:in invoke' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:141:ininvoke_task'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:in block (2 levels) in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:ineach'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:in block in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:108:inrun_with_threads'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:93:in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:71:inblock in run'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:158:in standard_exception_handling' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:68:inrun'
/var/lib/gems/1.9.1/gems/rake-10.0.2/bin/rake:37:in <top (required)>' /usr/local/bin/rake:23:inload'
/usr/local/bin/rake:23:in <main>' Tasks: TOP => assets:precompile:primary rake aborted! Command failed with status (1): [/usr/bin/ruby1.9.1 /usr/local/bin/rake ass...] /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils.rb:53:inblock in create_shell_runner'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils.rb:45:in call' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils.rb:45:insh'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils_ext.rb:40:in sh' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils.rb:80:inruby'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/file_utils_ext.rb:40:in ruby' /var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:12:inruby_rake_task'
/var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:21:in invoke_or_reboot_rake_task' /var/lib/gems/1.9.1/gems/actionpack-3.2.9/lib/sprockets/assets.rake:29:inblock (2 levels) in <top (required)>'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:in call' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:inblock in execute'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:in each' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:inexecute'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:166:in block in invoke_with_call_chain' /usr/lib/ruby/1.9.1/monitor.rb:211:inmon_synchronize'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:159:in invoke_with_call_chain' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:152:ininvoke'
/home/ubuntu-scheduler/Arbeitsfläche/codem-schedule/lib/tasks/codem.rake:56:in block (3 levels) in <top (required)>' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:incall'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:227:in block in execute' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:ineach'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:222:in execute' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:166:inblock in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:211:in mon_synchronize' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:159:ininvoke_with_call_chain'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:187:in block in invoke_prerequisites' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:185:ineach'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:185:in invoke_prerequisites' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:165:inblock in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:211:in mon_synchronize' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:159:ininvoke_with_call_chain'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/task.rb:152:in invoke' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:141:ininvoke_task'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:in block (2 levels) in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:ineach'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:99:in block in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:108:inrun_with_threads'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:93:in top_level' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:71:inblock in run'
/var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:158:in standard_exception_handling' /var/lib/gems/1.9.1/gems/rake-10.0.2/lib/rake/application.rb:68:inrun'
/var/lib/gems/1.9.1/gems/rake-10.0.2/bin/rake:37:in <top (required)>' /usr/local/bin/rake:23:inload'
/usr/local/bin/rake:23:in `

hi there,
i don't really know how to debug this.
after some time, the scheduler website ( overview/jobs/presets and hosts )
becomes very slow and unresponsive,

also the call to api/schedule url becomes very slow.

i have three hosts connected to the scheduler, all of which are available.

i tried reducing the call to api/schedule from every minute to every 5 minutes, but still after some time it happens.

i had this issue with the self installed version ( pre june 2 ) as well as with the debian package.

Load is near 0,
Plenty of Ram avaIlable

anyone has an idea how to resolv this?

If you want transcodem to generate thumbnails for a video, you have to specify a variable (%03d) in your output filename. This will make ffmpeg generate a bunch of stills, but this fails because transcodem doesn't keep the variable in the output filename.

Reproduce:

• preset "thumbs" with content "-r 1/10 -s 160x90"
• input: /tmp/test.avi
• output: /tmp/test-thumb%03d.jpg

Hi,
I setup a codem-transcode and added it to codem-schedule using web panel. Both servers were working fine until I stop codem-tanscode server and tried to go to hosts details page http://localhost:3000/hosts and web panel just stuck. I cant go to any other panel clicking the corresponding link. In Terminal I saw that server got freeze on a query SELECT "hosts".* FROM "hosts". Same thing happens when I stop the scheduler and start it again without starting codem-transcode server. When I delete the db and re-migrate then web panel starts working again

When finishing a job the final state is sometimes logged twice.

Sample output from log:

Scheduled   less than 5 seconds ago 
Accepted    less than 5 seconds ago The transcoder accepted your job.
Failed  less than 5 seconds ago ffmpeg finished with an error: ', 25.00 fps(r)Unknown codec 'libfaac' ' (1).
Failed  less than 5 seconds ago ffmpeg finished with an error: ', 25.00 fps(r)Unknown codec 'libfaac' ' (1).

First of all, congratulations with the nice app!

When trying to add a new job (both through curl and browser) an error is thrown stating:

Started POST "/api/jobs" for 127.0.0.1 at 2011-09-27 17:54:21 +0200
Processing by Api::JobsController#create as
Parameters: {"input"=>"/home/pieter/kiss.avi", "output"=>"/tmp/bar.mp4", "preset"=>"default"}
Completed in 38ms

NoMethodError (You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split):
app/models/job.rb:37:in from_api' app/controllers/api/jobs_controller.rb:96:increate'

Although this is probably an issue on my end, I just wanted to inform..just in case.

When I try to create a job via API, I get validation error saying all attributes are missing.

$ curl -X POST -H "Content-Type: application/json" -d '{"input": "/tmp/foo.flv", "output": "/tmp/bar.mp4", "preset": "720_MP4"}' http://localhost:4000/api/jobs.json

{"errors":{"source_file":["can't be blank"],"destination_file":["can't be blank"],"preset_id":["can't be blank"]}}

Here is a log for this request.

I, [2014-08-13T19:35:25.166434 #4194]  INFO -- : Processing by Api::JobsController#create as JSON
I, [2014-08-13T19:35:25.166598 #4194]  INFO -- :   Parameters: {"input"=>"/tmp/foo.flv", "output"=>"/tmp/bar.mp4", "preset"=>"720_MP4", "job"=>{}}

There is an empty job parameter added by wrap_parameters feature in rails I guess.

Unfortunately method Job.from_api will use this empty hash.

I am Happy and thank to this project .

since the ffmpeg support HLS , but codem-schedule cant apply with preset to transcode HLS format .

workaround for HLS ,each file to transcode HLS have to create it's own preset to fill in the ffmpeg option.

Currently the codem script doesn't check if the request is successful (for posting a new job). Should be better if it does and can retry it if possible.

Everything bigger then 2GB will be reported as 2GB once transcoding is complete. The transcoder sends the correct filesize, but the scheduler truncates that to 2GB because it uses int(11) (max value: 2147483647) and not bigint on MySQL.

When using ffmpeg from the command line the first command works. Since hls encoding creates multiple segments from the video files I am having trouble getting this to work. The code only seems to support a single output. Can this feature be added to the repo?

Command Line
ffmpeg -i LM.mp4 -vb 2500k -minrate 2500k -maxrate 2500k -bufsize 2500k -vf "scale='min(iw,1280)':-1" -s 1280x720 -acodec libfaac -ab 128000 -ar 44100 -ac 2 -vcodec libx264 -sn -r 29.97 -map 0 -f segment -segment_time 10 -segment_list playlist.m3u8 -segment_format mpegts -vbsf h264_mp4toannexb -flags -global_header 720p-%d.ts -threads 0

Preset
I entered this string as a preset in the scheduler.
ffmpeg -i LM.mp4 -vb 2500k -minrate 2500k -maxrate 2500k -bufsize 2500k -vf scale='min(iw,1280)':-1 -s 1280x720 -acodec libfaac -ab 128000 -ar 44100 -ac 2 -vcodec libx264 -sn -r 29.97 -map 0 -f segment -segment_time 10 -segment_list playlist.m3u8 -segment_format mpegts -vbsf h264_mp4toannexb -flags -global_header 720p-%d.ts -threads 0

The way the schedule endpoint works it breaks down when you have large amounts of jobs scheduled (10k+). The Schedule.run! method retrieves all scheduled jobs from the database and tries to schedule them regardless of how many slots are actually available.

Proposed change is to split into two phases: updating the jobs that are Processing or OnHold. After that scheduling jobs, but only retrieve as many from the database as there are slots available.

def run!
  jobs = to_be_updated_jobs
  jobs.each do |job|
    job.with_lock(true) do
      update_job(job)
    end
  end

  size = jobs.size

  jobs = to_be_scheduled_jobs
  jobs.each do |job|
    job.with_lock(true) do
      schedule_job(job)
    end
  end

  size + jobs.size
end

def to_be_updated_jobs
  Job.where(state: [Job::Processing, Job::OnHold])
end

def to_be_scheduled_jobs
  Job.where(state: Job::Scheduled).order('created_at').limit(available_slots)
end

(this can be written more cleanly though)

If no transcoders are available (down) but have been configured in the scheduler requesting the home and jobs pages can take quite a while and ultimately the entire app becomes unresponsive/unavailable.
Transcoder availability should not have any impact on whether the scheduler will correctly render pages or not.

job notifications are fired off before the tmp file transfers are written and the jobs table is updated. Is there a way to set the notifications to happen "after" everything is done, not just when ffmpeg finishes?

The RestClient timeouts as defined in transcoder.rb#68 do not work. The timeouts are only honored if you call RestClient.execute directly (or RestClient::Request.execute). Currently the scheduler code uses the send method to call the RestClient convenience methods (RestClient.get, RestClient.post, etc.). These do not take the timeout parameters into account, therefore you can get in a state where the hosts screen no longer works.

The order in which jobs are sent to the transcoders doesn't make sense. It should always prefer the oldest job (earliest submitted) first. Currently when there are a lot of jobs waiting it just seems to randomly pick one.

I would like to be able to use a FFMPEG option line like this :

-y -c:a aac -strict experimental -ac 1 -b:a 32k -ar 22050 -c:v libx264 -pix_fmt yuv420p -profile:v baseline -level 13 -b:v 200K -r 12 -g 36 -f hls -hls_time 10 -hls_list_size 999 -s 320x180 ts/320x180_.m3u8

This is for creating HLS compliant segmented h.264 with m3u8 playlist.
This works fine on the command line in Linux, however when using this as a preset in Scheduler it fails. (command line testing done on the same machine codem-transcoder is running on)
The main issue seems to be that the Scheduler API does not accept jobs without an "-o outputfile.xxx" , which now causes all kinds of strange things, like ffmpeg trying to mix 2 seperate streams.

N.B. Scheduler and Codem work fine in my situation when just using single Webm, h264 or Theora. It is just the segmented HLS that doesn't work.

• FFMPEG -> latest from Git
• Debian Wheezy 7.1
• codem & schedudler -> latest from Git

Hello
I have a web application that needs to get state of transcoding and progress number
I need some script or binary to call and get the stats
How can I get it?

A detailed overview of the state of the transcoders would be nice, preferably in the Hosts overview screen.

I 'm trying to use multiple hosts in a scheduler but they fail not finding the input. I know I could use nfs to mount the partition with same names across the hosts, but I'm wondering how is it supposed to be done ? I could not find in any doc.

many thanks in advance

I deleted a preset which was in use by one job. After deleting that preset I can no longer access the jobs page ("/jobs"). Gives 500 error (generic Rails error).

Using Debian Wheezy, also tried Ubuntu 12.04 LTS and older (1.9) version of Ruby :

output of rake codem:install

root@transcoder:~/codem-schedule# rake codem:install
rake aborted!
uninitialized constant Capistrano::Configuration::SSHKit
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano-3.0.0/lib/capistrano/configuration/server.rb:4:in <class:Configuration>' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano-3.0.0/lib/capistrano/configuration/server.rb:3:inmodule:Capistrano'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano-3.0.0/lib/capistrano/configuration/server.rb:2:in <top (required)>' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano-3.0.0/lib/capistrano/configuration.rb:3:inrequire_relative'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano-3.0.0/lib/capistrano/configuration.rb:3:in <top (required)>' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano_colors-0.5.5/lib/capistrano_colors.rb:1:inrequire'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/capistrano_colors-0.5.5/lib/capistrano_colors.rb:1:in <top (required)>' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:inrequire'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in block (2 levels) in require' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:ineach'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in block in require' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:ineach'
/usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in require' /usr/local/rvm/gems/ruby-2.0.0-p247/gems/bundler-1.3.5/lib/bundler.rb:132:inrequire'
/root/codem-schedule/config/application.rb:7:in <top (required)>' /root/codem-schedule/Rakefile:4:inrequire'
/root/codem-schedule/Rakefile:4:in <top (required)>' /usr/local/rvm/gems/ruby-2.0.0-p247/bin/ruby_executable_hooks:15:ineval'
/usr/local/rvm/gems/ruby-2.0.0-p247/bin/ruby_executable_hooks:15:in `

When i try to follow the job number link, i got an error

I'm on Centos7

This is the full trace:

activerecord (4.1.1) lib/active_record/associations/has_many_association.rb:79:in cached_counter_attribute_name' activerecord (4.1.1) lib/active_record/associations/has_many_association.rb:75:inhas_cached_counter?'
activerecord (4.1.1) lib/active_record/associations/has_many_association.rb:60:in count_records' activerecord (4.1.1) lib/active_record/associations/collection_association.rb:294:insize'
activerecord (4.1.1) lib/active_record/associations/collection_proxy.rb:729:in size' app/views/jobs/show.html.erb:28:inblock in _app_views_jobs_show_html_erb___722859217307401722_32261660'
activerecord (4.1.1) lib/active_record/relation/delegation.rb:46:in each' activerecord (4.1.1) lib/active_record/relation/delegation.rb:46:ineach'
app/views/jobs/show.html.erb:15:in _app_views_jobs_show_html_erb___722859217307401722_32261660' actionview (4.1.1) lib/action_view/template.rb:145:inblock in render'
activesupport (4.1.1) lib/active_support/notifications.rb:161:in instrument' actionview (4.1.1) lib/action_view/template.rb:339:ininstrument'
actionview (4.1.1) lib/action_view/template.rb:143:in render' actionview (4.1.1) lib/action_view/renderer/template_renderer.rb:55:inblock (2 levels) in render_template'
actionview (4.1.1) lib/action_view/renderer/abstract_renderer.rb:38:in block in instrument' activesupport (4.1.1) lib/active_support/notifications.rb:159:inblock in instrument'
activesupport (4.1.1) lib/active_support/notifications/instrumenter.rb:20:in instrument' activesupport (4.1.1) lib/active_support/notifications.rb:159:ininstrument'
actionview (4.1.1) lib/action_view/renderer/abstract_renderer.rb:38:in instrument' actionview (4.1.1) lib/action_view/renderer/template_renderer.rb:54:inblock in render_template'
actionview (4.1.1) lib/action_view/renderer/template_renderer.rb:62:in render_with_layout' actionview (4.1.1) lib/action_view/renderer/template_renderer.rb:53:inrender_template'
actionview (4.1.1) lib/action_view/renderer/template_renderer.rb:17:in render' actionview (4.1.1) lib/action_view/renderer/renderer.rb:42:inrender_template'
actionview (4.1.1) lib/action_view/renderer/renderer.rb:23:in render' actionview (4.1.1) lib/action_view/rendering.rb:99:in_render_template'
actionpack (4.1.1) lib/action_controller/metal/streaming.rb:217:in _render_template' actionview (4.1.1) lib/action_view/rendering.rb:82:inrender_to_body'
actionpack (4.1.1) lib/action_controller/metal/rendering.rb:32:in render_to_body' actionpack (4.1.1) lib/action_controller/metal/renderers.rb:32:inrender_to_body'
actionpack (4.1.1) lib/abstract_controller/rendering.rb:25:in render' actionpack (4.1.1) lib/action_controller/metal/rendering.rb:16:inrender'
actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:41:in block (2 levels) in render' activesupport (4.1.1) lib/active_support/core_ext/benchmark.rb:12:inblock in ms'
/usr/local/rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/benchmark.rb:303:in realtime' activesupport (4.1.1) lib/active_support/core_ext/benchmark.rb:12:inms'
actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:41:in block in render' actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:84:incleanup_view_runtime'
activerecord (4.1.1) lib/active_record/railties/controller_runtime.rb:25:in cleanup_view_runtime' actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:40:inrender'
actionpack (4.1.1) lib/action_controller/metal/implicit_render.rb:10:in default_render' actionpack (4.1.1) lib/action_controller/metal/implicit_render.rb:5:insend_action'
actionpack (4.1.1) lib/abstract_controller/base.rb:189:in process_action' actionpack (4.1.1) lib/action_controller/metal/rendering.rb:10:inprocess_action'
actionpack (4.1.1) lib/abstract_controller/callbacks.rb:20:in block in process_action' activesupport (4.1.1) lib/active_support/callbacks.rb:113:incall'
activesupport (4.1.1) lib/active_support/callbacks.rb:113:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:229:inblock in halting'
activesupport (4.1.1) lib/active_support/callbacks.rb:229:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:229:inblock in halting'
activesupport (4.1.1) lib/active_support/callbacks.rb:166:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:166:inblock in halting'
activesupport (4.1.1) lib/active_support/callbacks.rb:166:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:166:inblock in halting'
activesupport (4.1.1) lib/active_support/callbacks.rb:166:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:166:inblock in halting'
activesupport (4.1.1) lib/active_support/callbacks.rb:86:in call' activesupport (4.1.1) lib/active_support/callbacks.rb:86:inrun_callbacks'
actionpack (4.1.1) lib/abstract_controller/callbacks.rb:19:in process_action' actionpack (4.1.1) lib/action_controller/metal/rescue.rb:29:inprocess_action'
actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:31:in block in process_action' activesupport (4.1.1) lib/active_support/notifications.rb:159:inblock in instrument'
activesupport (4.1.1) lib/active_support/notifications/instrumenter.rb:20:in instrument' activesupport (4.1.1) lib/active_support/notifications.rb:159:ininstrument'
actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:30:in process_action' actionpack (4.1.1) lib/action_controller/metal/params_wrapper.rb:250:inprocess_action'
activerecord (4.1.1) lib/active_record/railties/controller_runtime.rb:18:in process_action' actionpack (4.1.1) lib/abstract_controller/base.rb:136:inprocess'
actionview (4.1.1) lib/action_view/rendering.rb:30:in process' actionpack (4.1.1) lib/action_controller/metal.rb:195:indispatch'
actionpack (4.1.1) lib/action_controller/metal/rack_delegation.rb:13:in dispatch' actionpack (4.1.1) lib/action_controller/metal.rb:231:inblock in action'
actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:in call' actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:indispatch'
actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:48:in call' actionpack (4.1.1) lib/action_dispatch/journey/router.rb:71:inblock in call'
actionpack (4.1.1) lib/action_dispatch/journey/router.rb:59:in each' actionpack (4.1.1) lib/action_dispatch/journey/router.rb:59:incall'
actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:676:in call' rack (1.5.4) lib/rack/etag.rb:23:incall'
rack (1.5.4) lib/rack/conditionalget.rb:25:in call' rack (1.5.4) lib/rack/head.rb:11:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/params_parser.rb:27:in call' actionpack (4.1.1) lib/action_dispatch/middleware/flash.rb:254:incall'
rack (1.5.4) lib/rack/session/abstract/id.rb:225:in context' rack (1.5.4) lib/rack/session/abstract/id.rb:220:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:560:in call' activerecord (4.1.1) lib/active_record/query_cache.rb:36:incall'
activerecord (4.1.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in call' activerecord (4.1.1) lib/active_record/migration.rb:380:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/callbacks.rb:29:in block in call' activesupport (4.1.1) lib/active_support/callbacks.rb:82:inrun_callbacks'
actionpack (4.1.1) lib/action_dispatch/middleware/callbacks.rb:27:in call' actionpack (4.1.1) lib/action_dispatch/middleware/reloader.rb:73:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/remote_ip.rb:76:in call' actionpack (4.1.1) lib/action_dispatch/middleware/debug_exceptions.rb:17:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/show_exceptions.rb:30:in call' railties (4.1.1) lib/rails/rack/logger.rb:38:incall_app'
railties (4.1.1) lib/rails/rack/logger.rb:20:in block in call' activesupport (4.1.1) lib/active_support/tagged_logging.rb:68:inblock in tagged'
activesupport (4.1.1) lib/active_support/tagged_logging.rb:26:in tagged' activesupport (4.1.1) lib/active_support/tagged_logging.rb:68:intagged'
railties (4.1.1) lib/rails/rack/logger.rb:20:in call' actionpack (4.1.1) lib/action_dispatch/middleware/request_id.rb:21:incall'
rack (1.5.4) lib/rack/methodoverride.rb:21:in call' rack (1.5.4) lib/rack/runtime.rb:17:incall'
activesupport (4.1.1) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in call' rack (1.5.4) lib/rack/lock.rb:17:incall'
actionpack (4.1.1) lib/action_dispatch/middleware/static.rb:64:in call' rack (1.5.4) lib/rack/sendfile.rb:112:incall'
railties (4.1.1) lib/rails/engine.rb:514:in call' railties (4.1.1) lib/rails/application.rb:144:incall'
rack (1.5.4) lib/rack/lock.rb:17:in call' rack (1.5.4) lib/rack/content_length.rb:14:incall'
rack (1.5.4) lib/rack/handler/webrick.rb:60:in service' /usr/local/rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/webrick/httpserver.rb:138:inservice'
/usr/local/rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/webrick/httpserver.rb:94:in run' /usr/local/rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/webrick/server.rb:294:inblock in start_thread'

If I enter http://127.0.0.1:8080 the transcoder host will appear as "available", if I enter http://127.0.0.1:8080/ (trailing slash) it will be marked as "unavailable"

Hi,

I am trying to install the scheduler for the first time. So i followed the steps ,

1 .git clone git://github.com/madebyhiro/codem-schedule.git
its fine.

1. bundle install

it shows ,

bash: bundle: command not found

im using cent os 6.5

Helps would be appreciated

I have tried to setup transcodem on Ubuntu 11.10
I managed to build and rake the transcoder and scheduler. This took some effort since Ubuntu does not use the latest version of ruby gems so I had to override some settings. The documentation on what gems you really need (and which verisons).

I got through the building provcess. However, when I run the scheduler and then visit the 'hosts' page on the website, it says:

Unitialized Constant JSON
(error in app/models/transcoder.rb, regel 45)

which seems to imply that I do not have the right gems installed.
Note that I haven't confiured presets (and probably hosts, not sure where tp check this outside of the web page) yet, but the error doesn't seem to imply missing data, but a problem in finding a class.
Below a list of what is installed.

ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-linux]

Rails 3.0.7

*** LOCAL GEMS ***

abstract (1.0.0)
actionmailer (3.0.7, 2.3.14, 2.3.11)
actionpack (3.0.7, 2.3.14, 2.3.11)
activemodel (3.0.7)
activerecord (3.0.7, 2.3.14, 2.3.11)
activeresource (3.0.7, 2.3.14, 2.3.11)
activesupport (3.0.7, 2.3.14, 2.3.11)
addressable (2.2.6)
arel (2.0.10)
builder (2.1.2)
bundle (0.0.1)
bundler (1.0.21)
crack (0.3.1)
diff-lcs (1.1.3)
erubis (2.6.6)
factory_girl (2.3.2)
factory_girl_rails (1.4.0)
i18n (0.5.0)
jquery-rails (1.0.19)
json (1.6.3)
json_pure (1.6.3)
kaminari (0.12.4)
mail (2.2.19)
mime-types (1.17.2)
multi_json (1.0.4)
polyglot (0.3.3)
rack (1.2.4, 1.1.2, 1.1.1)
rack-mount (0.6.14)
rack-test (0.5.7)
rails (3.0.7, 2.3.14, 2.3.11)
railties (3.0.7)
rake (0.9.2.2, 0.8.7)
rest-client (1.6.7)
rspec (2.7.0)
rspec-core (2.7.1)
rspec-expectations (2.7.0)
rspec-mocks (2.7.0)
rspec-rails (2.7.0)
simplecov (0.5.4)
simplecov-html (0.5.3)
sqlite3 (1.3.5)
thor (0.14.6)
treetop (1.4.10)
tzinfo (0.3.31)
webmock (1.7.8)
ZenTest (4.6.2)

I am admittedly a ruby/rails n00b. just wondering if there is an easy way to do this since I am running nginx on the same server as codem-schedule and it's wreaking havoc with my routes.

Hi,

When I create a given job with a post request, for instance:

headers: Content-Type: "application/x-www-form-urlencoded"
endpoint: http://MY_URL:3000/api/jobs
body:input=PATH_TO_INPUT&output=PATH_TO_OUTPUT_FILE&preset=MYPRESET&notify=MY_NOTIFY_URL

Everything works fine but the codem-schedule interface hangs and only responds after that given job status is turned to "complete" , the log as follow:

Started GET "/api/schedule" for 127.0.0.1 at 2018-12-03 18:29:01 -0200
Processing by Api::SchedulerController#schedule as */*
  Job Load (0.7ms)  SELECT `jobs`.* FROM `jobs` WHERE `jobs`.`state` IN ('processing', 'on_hold')  ORDER BY created_at
   (0.6ms)  BEGIN
  Job Load (10.8ms)  SELECT  `jobs`.* FROM `jobs` WHERE `jobs`.`id` = 6 LIMIT 1 FOR UPDATE
  Host Load (7.0ms)  SELECT  `hosts`.* FROM `hosts` WHERE `hosts`.`id` = 2 LIMIT 1
  SQL (0.5ms)  UPDATE `jobs` SET `progress` = 1.0, `duration` = 1, `filesize` = '1774012', `updated_at` = '2018-12-03 20:29:01' WHERE `jobs`.`id` = 6
  Job Load (0.5ms)  SELECT  `jobs`.* FROM `jobs` WHERE `jobs`.`id` = 6 LIMIT 1 FOR UPDATE
  StateChange Load (0.6ms)  SELECT  `state_changes`.* FROM `state_changes` WHERE `state_changes`.`job_id` = 6 AND (`state_changes`.`position` IS NOT NULL)  ORDER BY `state_changes`.`position` DESC LIMIT 1
  SQL (0.4ms)  INSERT INTO `state_changes` (`state`, `message`, `job_id`, `created_at`, `updated_at`, `position`) VALUES ('success', 'ffmpeg finished succesfully.', 6, '2018-12-03 20:29:01', '2018-12-03 20:29:01', 3)
  SQL (0.5ms)  UPDATE `jobs` SET `state` = 'success', `completed_at` = '2018-12-03 20:29:01', `message` = 'ffmpeg finished succesfully.', `updated_at` = '2018-12-03 20:29:01' WHERE `jobs`.`id` = 6
  Notification Load (18.7ms)  SELECT `notifications`.* FROM `notifications` WHERE `notifications`.`job_id` = 6
  StateChange Load (0.8ms)  SELECT  `state_changes`.* FROM `state_changes` WHERE `state_changes`.`job_id` = 6  ORDER BY position DESC LIMIT 1
  SQL (0.4ms)  INSERT INTO `deliveries` (`state`, `state_change_id`, `notified_at`, `notification_id`, `created_at`, `updated_at`) VALUES ('failed', 30, '2018-12-03 20:30:02', 5, '2018-12-03 20:30:02', '2018-12-03 20:30:02')
   (2.5ms)  COMMIT
  Host Load (0.4ms)  SELECT `hosts`.* FROM `hosts`
   (0.2ms)  BEGIN
  SQL (0.4ms)  UPDATE `hosts` SET `status_updated_at` = '2018-12-03 20:30:02', `updated_at` = '2018-12-03 20:30:02' WHERE `hosts`.`id` = 1
   (9.0ms)  COMMIT
   (0.2ms)  BEGIN
  SQL (0.4ms)  UPDATE `hosts` SET `status_updated_at` = '2018-12-03 20:30:02', `updated_at` = '2018-12-03 20:30:02' WHERE `hosts`.`id` = 2
   (3.2ms)  COMMIT
   (0.2ms)  BEGIN
  SQL (0.4ms)  UPDATE `hosts` SET `status_updated_at` = '2018-12-03 20:30:02', `updated_at` = '2018-12-03 20:30:02' WHERE `hosts`.`id` = 3
   (3.1ms)  COMMIT
  Job Load (0.5ms)  SELECT  `jobs`.* FROM `jobs` WHERE `jobs`.`state` = 'scheduled'  ORDER BY created_at LIMIT 18
Completed 200 OK in 60195ms (Views: 0.2ms | ActiveRecord: 62.1ms)

The notifications of the jobs works perfectly and the most strange thing: When I create a job with the codem script, that does not happen.

Thanks in advance