cyberboy666 / r_e_c_u_r

an open diy py/pi based video sampler

r_e_c_u_r

an open diy video sampler

r_e_c_u_r is an embedded python/openframeworks application on raspberry pi3 that uses input from the keypad to control video while displaying a simple text ui on a rpi lcd screen

in-depth video demo

features

• seamlessly loop video through rpi's HDMI or composite out
• intuitively browse video files from external and internal disk and map them into r_e_c_u_r
• load and trigger video samples from numbered slots in the sampler bank
• dynamically set and clear the start/end points of each sample as it plays
• control and sequence all inputs and more with midi-usb
• many sampler modes for varied playback including: repeat, one-shot, gated, random, fixed-length, random-start and more
• exhaustive and extendable settings menu to suit your use

extensions

as of V2.0 the r_e_c_u_r platform has been extended in a number of directions to complement live performance and _recur_ , the original video playing engine:

• _captur_ : input live video via CSI or USB , for live sampling and processing
• _conjur_ : glsl shader engine, map and trigger shader files and control parameters in real time; generate video or process and mix existing sources
• _detour_ : in-memory frame sampling (up to 500 frames), video scrubbing, playback speed and direction control, finger drumming
• toggling internal feedback + strobe effect

NOTE: most of the new V2.0 features are optimised for sd composite video output. Some of these extensions will work over HDMI but they are not fully supported - especially the live video input struggles

extension video walkthroughs

• 
• 
• 

main objectives:

• Affordable : reducing the entry cost to performing with video
• Extendable : laying the foundations (of a user interface and code style) that can be easily iterated on by the community
• Simple : easy to operate (abstracted completely from ‘driving’ a raspi ) , easy to build (no technical computer install-y or circuit-y knowledge required to diy) , easy to develop (human readable code, inviting amateur/first time coders to contribute)

documentation:

• operating - how to use r_e_c_u_r
• building - how to diy r_e_c_u_r
• developing - how to contribute to r_e_c_u_r

many other things documented on the wiki

status

The nature of this project is to be open-ended and community driven. my r_e_c_u_r already solves the problems i initially built it for. what happens next depends on how it is used and received by you. if you like the idea please let me know / get involved !

contact, donation and thanks

[email protected]

you can find me (and others) at https://scanlines.xyz , an online diy av community space. theres a r_e_c_u_r channel in the chatroom: https://chat.scanlines.xyz/channel/r_e_c_u_r there is also a facebook user group : https://www.facebook.com/groups/114465402691215/

all feedback is appreciated. if you want to donate to this project you can do so with the above email via paypal : everything i receive will go into improving r_e_c_u_r. cheers to Leo Browning for the 3d modelling and vector art and to Ben Caldwell for heaps of help with the code!

r_e_c_u_r's Issues

export a working image of raspberrian/r_e_c_u_r/for-pi3 and use it to flash new pi

also take a fresh raspberian and configure to a working recur by following instructions in dotfile..

create documentation for operating

my rasbPi has root account locked - boot in sulogin..

basically this exact problem, i have a feeling it has to do with booting with the usbdrive in the pi, not sure tho...

hide boot text and use custom splash screen ?

info on this here https://raspberrypi.stackexchange.com/questions/59310/remove-boot-messages-all-text-in-jessie

seek buttons through errors

remove subprocess shell=True calls

various settings and options require the python program to interact with other programs on the pi. apparently its not good practice to use the shell=True , but for some of the more complicated calls i couldnt figure out how to call them correctly.
it is a low priory right now but i should go back and do this properly sometime !

recur handling openframeworks crash

the openframeworks backend sometimes crashes, this seems to be if it tries an action on a video in a format it doesnt like but there could be other reasons ?

• it would be good to make the of app more resilient to this kindof thing
• failing that it would be good if recur could 'detect' this or atleast quit the crashed process and start a new one ...

load key mappings from json

Different play modes including : random, sequence, random start, fixed length etc

some ideas for playback modes :

• random : loads a random (non-empty) slot from current bank for next playback
• sequence : allows you to key in a combination of numbers to loop between in current bank
• random start : loads the next video at a random start point
• fixed length : always plays videos for the same amount of time (input from BPM tap or something)

create, test and upload v2 of 3d printed panels

some things to consider for v2 of panels:

• making the panels thicker
• slightly wider to fix one end of keypad...
• main support holes slightly bigger for screw size
• more support pillars ?
• better screw size? (that doesnt overhang through the top?)
• add small screw holes for raspberry pi
• [ ]spacers between screen and pi
• fitted to the keyboard? a clamp to remove needing to screw in keypad

other vague panel-y things to think about;
how might we design/modify the panels for:

• eurorack format ? breakout connections of pi inputs?
• with extra height/inputs of piCaptureHat ?
• no built in keypad, strictly midi inputs/ a usb a keyboard ?
• other sbc such as orange pi / pi2 etc ?

changing key mappings on every boot making start up slow

currently , we need to press enter on boot. (due to issue #37 )

then after every boot it maps the keys, however this takes up to 4 seconds , which slows loading time. idealy we wont need to enter on boot (or could have that key press do both enter and our key in meantime) so we can keep the custom mapping until we tell it otherwise

create sticker template to relabel numpad keys

simple symbols and characters, white on black background, with smaller text at bottom describing second function. shared in cloud-doc format for easy customization

rename all usage of 'bank' with 'slot' to avoid confusion

fix the error handling

currently there is some attempt to display info and error messaging on the screen but it needs some work and testing. also should write any 'unexpected' errors to a log file for troubleshooting

display doesn't refresh on key press

swirches to hdmi output when should be composite

write some tests !

its getting to the point where having tests over this project would be v useful !

• have a view of the code coverage
• tests over the actions
• unit tests closer to some pathes
• changing things breaks the test

tidy up settings tab to not display non-existence features and to be more intuitive

Add tags to make this easily discoverable on github

Just realised how important tags are on github when people are searching for repositories.
Would be a good idea to put them in if you are interested in this :)

creating/accessing different banks

at this stage im thinking of keeping this as simple as possible :

• each 'bank' has 10 slots. (a video can be refered to as bank.slot )
• the current display_data.json file will have a list that is appended to
• three new actions : cycle_bank_forward, cycle_bank_back, clear_current_bank
• if your highest bank is not empty then auto create a new empty highest bank. if your second highest bank is clear then auto delete the highest bank.

some code refactors

• need a place for state variables of the system that can be accessed everywhere (kinda weird that the display_mode and control_mode currently live in display class, also needing to access the settings.json without going through data object maybe ?)
• similar but for helper methods ie read_json
• some of the depandances around the data area need some work

adjusting video playback speed

on <SAMPLR> mode , an extra column that displayed the playback speed : 1.0 by default ? would have to experiment with max and min values and if different file types / formats have the same or different limits.

a base action : change_speed_by(value) which can be called by parent actions in any of the ways described below

a placeholder video for when all banks are clear maybe ?

some keys on the num pad are not usable on device

the numlock key and the 000 key have a specific operation that makes it difficult to map within the program. a custom keyboard layout will need to be created on machine to strip these functions and make it easier.

make the 000 key mapable by distinguishing between multiple presses

update position variables from integers to floats

start, end and duration need to be updated to floats for more control, rounding up 0.5s could create issues when checking for the end of a clip

also need to add length variable which is the difference between the start and end points

does not store/display length of video

when a video is mapped to a slot it should get the video length and display it

make a dotfiles directory

The purpose of a dotfiles directory being to document the necessary configuration changes made to the pi in order to run the project.

The directory can include the actual config files such as

-.remap
-launcher.sh
-.config/xdg/lxsessions.....autostart
-drivers for the screen
-python packages necessary for installation

video player can crash when triggering an empty slot

fix dev mode

when switching dev on mode the pi reboots with display on hdmi out, when off it reboots to display again

fit/fix the display output to screen size

the browser and settings mode only shows 5 lines although the screen can fix more. also the sampler mode has 15 slots in a bank where theres only keys for 10

create documentation for building

work in progress - see here : build documentation

add git integration

• display branch name in title
• action to pull latest version/switch branch

modifying the setting tab to toggle values AND evoke actions

currently settings is used to toggle things - ie change the screen resolution or switch playback mode etc. going forward we may want to trigger an action (ie quit or restart the program, switch to hdmi out). we also could still access key presses that arnt mapped on keyboard - allowing more flexibility around custom mappings, shortcuts and actions...

need to configure the pi to run on boot

some hdmi displays struggle with 1080video and the pi struggles with 1080hdmi output

for hdmi devices whose preferred hdmi setting is 1080 , recur seems to drop the video for all filetypes.
it seems to be related to the memory being used (without recur running , two videos is plays fine but three causes same dropout) but changing the gpu/cpu split etc doesnt help.

what does help it is dropping the hdmi output setting down to 720. from here recur will play all videos fine including 1080video files.

while investigating this i also noticed that my hdmi-vga converter (which prefers DMT x DVI) also drops while trying to play 1080 videos. maybe just because it cant handle the scaling etc ?

duplicate usb folders loaded

sometimes the pi loads my usb as 'TIM1' and has a 'TIM' folder stuck in pi/media.

check if mapped videos still exist

it would be useful if, on startup, the program checked that it could locate each file mapped to a slot. if it couldnt find it, should still display it but grayed out.

support video file types other than mp4

a thorough testing of the video player with respect to a number of variables would be useful including :

• file type (container and codex ? mp4 vs mkv etc)
• file length (how does 4 hour videos play ? is seeking slower ? )
• file resolution (what happens with 1080p files ?)

sometimes gets stuck loading first video

fix navigation highlighting

the length of selected row is inconsistent

also (internally) the browser index should be absolute rather than relative

also setting and browser navigation seem dependant

recording video input for live sampling

an optional extension : c_a_p_t_u_r - for live video sampling.

• uses input from raspberry pi camera (or PiCapture boards)
• a new action : toggle_recording
• when recording is on, display [recxx:xx] between the now and next on video_display panel.
• save the file to flashdrive (or pi?) and auto map to a bank
• have a display mode for monitoring video input, and displaying/switching input channels ...

option to copy videos from device onto recur sd

since most sd cards are 16g or greater , there is plenty of unused space that could store the videos you wanted to perform with , making it easier.

• would need to check the available space on the sd card before copying.
• maybe a second function of the browser menu
• options to copy files or whole folders

usb not showing when inserted after boot

i started up recur , and then plugged in the usb. the files already mapped could be loaded and played fine but the browser mode had not refreshed to reflect this. just need a way to tell the browser list to redraw (if u re-enter the browser screen perhaps ?)

browser slot number is not updating

when a file is mapped to a slot the browser column does not update with the new slot number

play placeholder video when all banks are clear

have a video file saved to the pi that loads as placeholder when banks are clear. also could load a tutorial video for peoples first time

small memory leak in display code

while fixing a noticeable memory leak in the display code (turns out the tk text object stores a list of every state in has been in by default, which is bad if u update the object every half second !) i discovered another much smaller leak in the same area of the code (if i stop the display updating there is no noticeable leaks at all).

when i set the display to update every millisecond , i can see the size of the python process slowly increasing.

im not sure what the problem is atm , i added a reset option in the settings as a workaround if memory maxs out.

v3 of the enclosure

some changes to make to v2 (green) enclosure:

1. align the raspi slightly under the panel
2. raise the pi up slightly (?)
3. tighten the keypad horizontally
4. adjust the height of keypad platform slightly to fit without rubber feet (?)
5. resize the keypad platform to fit better and reduce print time (?)
6. print in black

the video (over HDMI) does not adjust to fit screen

when recur is connected to screens with different resolution it should adjust to fit the screen

have a setting that is screen resolution - this can be toggled between the ones i encounter and can be added to for other cases

compromised video playback on some hdmi displays

• i have been using an hdmi-to-vga dongle to test and dev recur on an old vga computer monitor.

with this setup the video playback is exactly as expected, however when i try running recur on a real hdmi the video playback experiences some problems :

• it sometimes freezes the playback a few seconds into playing the video
• when loading the next video it interrupts the current video and gets stuck displaying first frame of next video rather than finishing what it is playing.

i have spend some time trying to figure out why it seems to work for my hdmi-vga dongle, including trying different (lower) screen resolutions but no luck.

this is the biggest and most crippling bug discovered , if this cant be resolved , i couldnt offer hdmi out from this device