turion / essence-of-live-coding

Universal Live Coding & Functional Reactive Programming Framework

README

---

essence-of-live-coding is a general purpose and type safe live coding framework in Haskell. You can run programs in it, and edit, recompile and reload them while they're running. Internally, the state of the live program is automatically migrated when performing hot code swap.

The library also offers an easy to use FRP interface. It is parametrized by its side effects, separates data flow cleanly from control flow, and allows to develop live programs from reusable, modular components. There are also useful utilities for debugging and quickchecking.

Change the program, keep the state!

Live programs

In essence, a live program consists of a current state, and an effectful state transition, or step function.

data LiveProgram m = forall s . Data s
  => LiveProgram
  { liveState :: s
  , liveStep  :: s -> m s
  }

We execute it by repeatedly calling liveStep and mutating the state. The behaviour of the program is given by the side effects in the monad m.

Example

Here is a simple example program that starts with the state 0, prints its current state each step and increments it by 1:

data State = State { nVisitors :: Int }

simpleProg = LiveProgram { .. }
  liveState = State 0
  liveStep  = \State { .. } -> do
    let nVisitors = nVisitors + 1
    print nVisitors
    return $ State { .. }

We can change the program to e.g. decrement, by replacing the body of the let binding to nVisitors - 1. It's then possible to replace the old program with the new program on the fly, while keeping the state.

Migration

The challenge consists in migrating old state to a new state type. Imagine we would change the state type to:

data State = State
  { nVisitors  :: Int
  , lastAccess :: UTCTime
  }

Clearly, we want to keep the nVisitors field from the previous state, but initialise lastAccess from the initial state of the new program. Both of this is done automatically by a generic function (see LiveCoding.Migrate) of this type:

migrate :: (Data a, Data b) => a -> b -> a

It takes the new initial state a and the old state b and tries to migrate as much as possible from b into the migrated state, using a only wherever necessary to make it typecheck. migrate covers a lot of other common cases, and you can also extend it with user-defined migrations.

Functional Reactive Programming

In bigger programs, we don't want to build all the state into a single type. Instead, we want to build our live programs modularly from reusable components. This is possible with the arrowized FRP (Functional Reactive Programming) interface. The smallest component is a cell (the building block of everything live):

data Cell m a b = forall s . Data s => Cell
  { cellState :: s
  , cellStep  :: s -> a -> m (b, s)
  }

It is like a live program, but it also has inputs and outputs. For example, this cell sums up all its inputs, and outputs the current sum:

sumC :: (Monad m, Num a, Data a) => Cell m a a
sumC = Cell { .. } where
  cellState = 0
  cellStep accum a = return (accum, accum + a)

Using Category, Arrow, ArrowLoop and ArrowChoice, we can compose cells to bigger data flow networks. There is also support for monadic control flow based on exceptions.

Use it and learn it

Setup and GHCi integration

For the full fledged setup, have a look at the gears example, or the tutorial project. The steps are:

• Create a new cabal project containing an executable (preferably in a single file), and add essence-of-live-coding as a dependency.

• There are backend packages available:

  What?	Which backend?	Which library?
  Sound (PCM)	PulseAudio	essence-of-live-coding-pulse
  Sound (Synthesizers)	Vivid & SuperCollider	essence-of-live-coding-vivid
  Sound (Midi)	PortMidi	essence-of-live-coding-PortMidi
  2d vector graphics	gloss	essence-of-live-coding-gloss
  Webserver	WAI	essence-of-live-coding-warp

  Some will require external libraries to link properly. The tutorial project shows you how to install those using Nix.

• There is a custom GHCi script that supply quick commands to start, reload and migrate the program automatically. You can usually copy it from the templates folder.

• Write a main program called liveProgram :: LiveProgram m, where m is a Launchable monad, such as IO.

• In a REPL:

  • Launch cabal repl.
  • Run your program with :livelaunch.
  • Edit your program, and reload with :livereload.

• Instead of reloading manually, you can use ghcid to do this manually for you.

  • Install ghcid.
  • Copy templates/.ghcid into your project folder.
  • Simply launch ghcid and your program will start.
  • Simply edit your file and the changes will reload as soon as they compile.

Examples

• The tutorial project.
• The gears example uses gloss for graphics and PulseAudio for sound.
• There is a demo using the WAI web server demos/app/DemoWai.
• The article contains numerous examples, for example a webserver, a sine generator, and a control flow example
• There are test cases that show how the automatic migration works precisely in certain examples.

Reading

• ICFP 2019 presentation. For a quick pitch.
• Abstract For a 2-page overview.
• Preprint article. For all the details.
• Appendix. For all the details that were left out in the article.
• The dunai and rhine packages. For backup material on how to program in this FRP dialect. (Spoiler: rhine is going towards live coding soon.)

Best practice

In order to get the best out of the automatic migration, it's advisable to follow these patterns:

• Use the FRP interface. (Section 4 in the article, LiveCoding.Cell) It builds up the state type in a modular way that is migratable well.
• Develop modularly. Your top level cell should not be a long arrow expression, but it should call separately defined arrows. This makes the state more migratable, which is useful when you edit only in the "leaves" of your arrow "tree".
• Wherever you write Cells from scratch, or use feedback, use records and algebraic datatypes to structure your state. (Not just tuples and Either.)
• Use ghcid in order to save yourself the hassle of having to reload manually all the time. Vanilla .ghci and .ghcid file templates can be found in the root directory, and usually it suffices to copy these two files into your project and launch ghcid from there.
• When the automatic migration would fail, there are user migrations (LiveCoding.Migrate.Migration). Often, it's good practice to wrap your state in a newtype first (migration to newtypes is automatic), and then migrate this newtype.
• Use exceptions for control flow. (Section 5 in the article) That way, you can keep the control state of your program when migrating.
• Don't keep state in concurrent variables such as MVars, IORefs, and so on, if you don't need to. It cannot be migrated well. (During a migration, the variable might be deleted, garbage collected, and reinitialised.) Instead, all migratable state should be inside Cells. The only use case for such a variable is an external device, resource, or thread, and in this case you should use a LiveCoding.Handle.

Known limitations

• For custom migrations, you currently need to write your own .ghci file.
• If your program doesn't compile when you enter GHCi, you will need to leave GHCi again and recompile. Otherwise the live coding infrastructure will not be loaded. After it has successfully compiled, you can reload from within GHCi, and further compilation errors do not affect the infrastructure.
• The template .ghci and .ghcid files assume that you are using a recent version of cabal (at least version 3). If this is not the case for you, consider updating, or adapt the files to use new-style commands.

Contributors

• Manuel Bärenz https://github.com/turion/
• Miguel Negrão https://github.com/miguel-negrao/

essence-of-live-coding's Issues

Refactor cellStep using StateT and lenses?

Recently (#57) it has come up again that in essence, the Cell constructor could also be declared as:

data Cell m a b = forall s . Data s => Cell
  { cellState :: s
  , cellStep :: a -> StateT s m b
  }

Is this a good idea?

Advantages

• Potential conceptual clarification
• Code reuse from transformers
  • This might resolve some strictness issues
  • Simplifies and clarifies a lot of implementations where state is manually threaded right now

Disadvantages

• One more level of indirection
• Tying to a specific transformer and library

Work to do

• Use lenses (https://hackage.haskell.org/package/lens-5.0.1/docs/Control-Lens-Zoom.html#v:zoom) to implement the different cell manipulation functions
• Check whether the article still works with such changes

Every commit should compile

I want to check on CI that every commit compiles (including tests). This could be done by adding a line like:

- git rebase origin/master --exec "cabal build all --enable-tests"

or so, but I need to test.

Flesh out webserver example into own backend package

Make ExceptT instance of Launchable?

Right now, we can't stop programs if they throw exceptions in ExceptT or MaybeT. We could mimick this behaviour by throwing IO exceptions.

Integration tests for backends

It would be nice to have integration tests that:

1. Launch a simple example program using one backend
2. Use an external tool to create a stimulus
3. Observe & react
4. Use an external tool to observe the reaction

For example when #54 (MIDI backend) is merged, we could:

1. Use an external tool like https://github.com/gbevin/SendMIDI to send some MIDI events
2. Transform them in an example live program (e.g. change the pitch)
3. Send them back
4. Observe the response using some external tool like aseqdump

Similar tests could be done for the other tests. Some might require a certain level of virtualization (possibly QEMU), which can be achieved by reusing e.g. the nixos integration test framework.

GHC versions 9.6 and 9.8

These should be supported.

Higher-order state support

Hello!

Essence-of-live-coding doesn't support higher order state, as stated in:

essence-of-live-coding/essence-of-live-coding/src/LiveCoding/Coalgebra.lhs

Line 88 in 279f42a

The price of \mintinline{haskell}{Data} is loss of higher-order state.

This means that higher order arrow constructs such as pSwitch aren't possible to implement with state remaining intact between reloads.

Unfortunately, for most games, one needs dynamically many entities, which I believe requires use of higher order arrows.

Using essence-of-live-coding to develop games would be amazing. Currently, my engine architecture makes use of essence-of-live-coding so that GPU initialization and windows can be kept between reloads, but the state simulation of my game has to rely on a library supporting higher order state, such as Dunai. At the moment, I record all inputs and then replay them in sequence to get a pseudo live coding environment, but the process of replaying past state quickly starts to take too long.

Is there any way around this limitation at all?

Thank you!
Sebastian

Test control flow migrations

The function migrationExceptState is not sufficiently tested yet.

Extend nonBlocking to UnliftIO, MonadConc or another IO abstraction

nonBlocking cells are introduced in #1. They are currently restricted to IO, but this should be generalised.

Move all example projects in own subfolder

The main repo looks a bit cluttered. It's unclear e.g. that gears is an example.

Add Selective instance to Cell

This is basically the same issue like ivanperez-keera/dunai#239.

See https://hackage.haskell.org/package/selective-0.4.1.1/docs/Control-Selective.html#t:Selective.

Reorder/refactor test utilities and essence-of-live-coding-quickcheck

Ideally, the tests in essence-of-live-coding should be a standard application of methods and utilities from essence-of-live-coding-quickcheck. Right now there is some duplication.

portMidiHandle being private prevents using PortMidi cells with cells with different exceptions

If PortMidi code needs to be used together with other code with different exceptions, then the exceptions need to be merged via

mapErrorC :: Monad m =>
    (e -> e') -> Cell (ExceptT e m) a b -> Cell (ExceptT e' m) a b
mapErrorC f = hoistCell (withExceptT f)

The fact that portMidiHandle is hidden prevents, in such a situation, creating a version of runPortMidiC that works with other exceptions since the exceptions being used are fixed. Would it be ok to export portMidiHandle ?

Add Profunctor instance to Cell

Short description

A Cell is a Profunctor.

Explanation

Given functions y -> a and b -> z, a Cell m a b can be transformed into a Cell m y z, by precomposing with the first function and postcomposing with the second one.
The step function of a cell has type cellStep :: s -> a -> m (b, s), so for postcomposition fmap is needed.

This shouldn't be to hard, so if you're looking for a way into the code base, feel free to give it a try :)

To do

• A dependency to the profunctor package should be added
• An instance Functor m => Profunctor (Cell m) should be added

Support GHC 9.0 and 9.2

HLint everything

• Call HLint and resolve all reasonable suggestions, and decide which to ignore
• Add HLint to CI

Get rid of literate Haskell?

While it was nice to try out literate Haskell, it's getting in the way of serious development. It's hard to keep an overview, ghcide doesn't support it properly (https://github.com/digital-asset/ghcide/issues/682), haddock and literate parts get in each other's ways.

Rough idea

Try to move all the actual code into proper *.hs files, possibly small ones, and quote it from LaTeX.

Properties to keep

• The article should still show the actual code
• The code should have good haddocks

Rough implementation ideas

• \inputminted with the firstline=n option, and \newmintedfile. Problem: If code changes, I have no way to see that, except by reading through the article.

Consider shipping own hot code swap

It would be possible to run interpreted code similar to this MWP: https://gist.github.com/gelisam/18ac5172f0c209705ae0339bd7711c43

Document ghcid setup

It's possible to let the reloading be done automatically with ghcid. This should be documented.

Gloss thread does not shut down gracefully

Reproduce

• Take e.g. essence-of-live-coding-gloss-example
• Launch it
• Set liveProgram = mempty
• Reload

Bug

• There will be a leftover window which doesn't close, or some crash, or some error
• Resetting the program to what it was before and reloading it doesn't gracefully restart

Use the Has type class instead of explicit monad transformer stacks

Would it make sense to always use the Has type class in all packages here instead of explicit monad transformer stacks ? It would be a bit of work converting all functions to use Has, and type signatures would be a bit more abstract, but on the positive side it would mostly eliminate the use of lifts in eolc-based code, right ?

Mark several GHC warnings as build failures

Certain GHC warnings should count as build failures. As a good measure:

• Enable -Wall
• Enable -Werror
• Try to fix everything, disable some warnings again if it's to hard to fix

At least, all -Wincomplete-* flags should be activated. (See e.g. #63)

See https://downloads.haskell.org/ghc/latest/docs/html/users_guide/flags.html#warnings for docs for all warnings.

Use arrow-utils

General utility functions for arrows should go to https://hackage.haskell.org/package/arrow-utils-0.1.0.0, and be reused here.

Document Finite

The type class Finite could use better documentation.

There is some motivation in section 3.1 of this appendix: https://www.manuelbaerenz.de/essence-of-live-coding/EssenceOfLiveCodingAppendix.pdf

But there is no documentation in the haddock. This could be improved.

Add Traversing instance to Cell

Follow-up to #56.

I think that Cells should be an instance of https://hackage.haskell.org/package/profunctors-5.6.2/docs/Data-Profunctor-Traversing.html#t:Traversing. Basically, this means that there should be a function traverse' :: Traversable f => Cell m a b -> Cell m (f a) (f b). For example, a cell can be lifted into a cell of lists, or a cell of trees.

To do

• Add Choice instance
• Add Strong instance
• Add Traversing instance, probably implementing wander and not traverse'

Support GHC 8.10

Use ResourceT

It seems resourcet does something similar to my Handle. I should refactor and make sure that typical ResourceT usage can be transformed into Handles in a simple way.

(Thanks @dpwiz for pointing this out!)

Improve the docs of the main module

Ideally, https://hackage.haskell.org/package/essence-of-live-coding-0.2.5/docs/LiveCoding.html gives a good entry point to the library. Right now it just vomits all imports concatenated.

• There should be sections corresponding to the modules (maybe these sections can be added in the modules themselves)
• There should be an intro and some examples
• Unimportant things like Data should be hidden or come last

Consider shifting the type parameters of CellExcept to make it a monad morphism

If we change the type parameters of CellExcept m a b e to CellExcept a b m e, we can easily write instances for MonadTrans and MFunctor. Possibly even for MMonad. This should be done in the next major release.

Use acid-state?

It would be great to not be stuck on the Data instance and use other ways of serializing the state. Then it would be possible to persist the state over different "backends". This is probably necessary to go beyond storing the state locally in a process. One way to serialize might be the IsAcidic class (https://hackage.haskell.org/package/acid-state-0.16.1.3/docs/Data-Acid.html#t:IsAcidic).

h-raylib backend

I have a hello world here. The gloss backend I started with is longer. Maybe those features are not needed for a larger h-raylib program?

Add nonblocking IO test

The test suite for NonBlocking (#1) isn't automated. Currently, one has to manually inspect the result. There should be a proper IO unit test.

Cells without data constraints on state

Hi! Is there any way to disable data migration for certain cell functions so state doesn't need the Data constraint? Specifically I need delay :: (Data s, Monad m) => s -> Cell m s s to become delay :: (Monad m) => s -> Cell m s s.

I need it because I'm having trouble defining data instances for some of my data types (due to some GADTs being difficult to derive data from). I don't mind them being reset to their default value on every migration.

I think I can do this using the Handle mechanism by storing the GADT in a MVar or Foreign.Store. That way I only need to store the mvar/store handle in the cell state which has a data instance. Is there an easier way of doing this? Thanks!

runExceptC is missing the ArrM case

The function LiveCoding.Exceptions.runExceptC is missing a case in the pattern match, for ArrM. This makes it crash when calling runExceptC on an arrM construction. The case should be added.

I could have found this by activating -Wincomplete-patterns -Werror I think.

Replace Monad instance for CellExcept by Selective?

The Monad instance for CellExcept comes at a technical cost, which is the awkward Finite class. I believe that CellExcept is a Selective in a natural way, and that I should try and see whether that is sufficient. The Finite trick can sort of be repeated with bindS.

fails with ghc-9.4 for Stackage Nightly

Building library for essence-of-live-coding-0.2.6..                                                                     [308/182284]
[ 6 of 41] Compiling LiveCoding.Cell.Monad                                                                                          
                                                                                                                                    
/var/stackage/work/unpack-dir/unpacked/essence-of-live-coding-0.2.6-1d9736c86ef45ccba8284a4881f4f3c3e0269ac83cbb9904143336a7082d2684
/src/LiveCoding/Cell/Monad.hs:48:6: error:                                                                                          
    • Could not deduce (base-4.17.0.0:Data.Typeable.Internal.Typeable                                                               
                          t)                                      
        arising from a superclass required to satisfy ‘Data (t s)’,                                                                 
        arising from a type ambiguity check for                                                                                     
        the type signature for ‘hoistCellKleisliStateChange’                                                                        
      from the context: (Monad m1, Monad m2,                                                                                        
                         forall s. Data s => Data (t s))                                                                            
        bound by the type signature for:                                                                                            
                   hoistCellKleisliStateChange :: forall (m1 :: * -> *) (m2 :: * -> *)
                                                         (t :: * -> *) a1 b1 a2 b2.                                                 
                                                  (Monad m1, Monad m2,                                                              
                                                   forall s. Data s => Data (t s)) =>                                               
                                                  (forall s.                                                                        
                                                   (s -> a1 -> m1 (b1, s))                                                          
                                                   -> t s -> a2 -> m2 (b2, t s))                                                    
                                                  -> (forall s. s -> t s)                                                           
                                                  -> Cell m1 a1 b1                                                                  
                                                  -> Cell m2 a2 b2                                                                  
        at src/LiveCoding/Cell/Monad.hs:(48,6)-(53,18)                                                                              
      or from: Data s                                                                                                               
        bound by a quantified context                                                                                               
        at src/LiveCoding/Cell/Monad.hs:(48,6)-(53,18)                                                                              
    • In the ambiguity check for ‘hoistCellKleisliStateChange’                                                                      
      To defer the ambiguity check to use sites, enable AllowAmbiguousTypes                                                         
      In the type signature:                                      
        hoistCellKleisliStateChange :: (Monad m1,                                                                                   
                                        Monad m2,                                                                                   
                                        (forall s. Data s => Data (t s))) =>                                                        
                                       (forall s.                                                                                   
                                        (s -> a1 -> m1 (b1, s)) -> (t s -> a2 -> m2 (b2, t s)))
                                       -> (forall s. (s -> t s)) -> Cell m1 a1 b1 -> Cell m2 a2 b2
   |                                                              
48 |   :: (Monad m1, Monad m2, (forall s . Data s => Data (t s)))                                                                   
   |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...         

ps Could I suggest using a https url for the git repo in .cabal rather than ssh?

Evaluate whether brick is a good backend

Quite a few games have been written with https://hackage.haskell.org/package/brick. Maybe there could be a good backend to it.

Test and improve performance

It would be great to introduce a few benchmarks (in particular in analogy to ivanperez-keera/dunai#370 and ivanperez-keera/dunai#375), and possibly adopt @lexi-lambda 's performance improvements in:

https://github.com/lexi-lambda/incremental/blob/267b83a8d69c95ea6070c99af9fc49106afbf166/src/Incremental/Fast.hs

The three optimization categories I can read off there are:

• Inlining all basic constructions (and inline . conlike)
• A builder function (inlined conlike) with rewrite rules to optimize the common case where the state type or the input type are ()
• Make cell/rule/machine state strict

Adapt get-tested?

There is an interesting tool for generating the github CI matrix from the cabal file: https://github.com/Kleidukos/get-tested/ Maybe this would be useful.

Support stack

• Set up CI to build the project with stack, in a few past snapshots
• Fix possible build errors

@miguel-negrao I likely won't work on this any time soon, but be my guest in case you want to submit a PR :)