mlajtos / l1

L1: Tensor Studio — The playground for tensors

L1: Tensor Studio

L1: Tensor Studio is a live-programming environment for differentiable linear algebra. The playground for tensors.

Live Demo | Documentation | Examples

About

L1 is a playground for differentiable linear algebra, heavily used in Machine Learning. It frees your mind from accidental complexities of programming, and lets you focus your attention on the underlying math to further strengthen your intuition.

Goal

Become the standard tool for prototyping new Machine Learning ideas. More...

Features

• designed for rapid learning and prototyping
• helpful live visualization
• elegant pure functional language
• eager execution
• auto-broadcast for tensor operations
• fast GPU-accelerated computation
• awesome built-in documentation
• syntax-highlighting and code-completion

Changelog

• August 2018

What's comming?

• variable tensors and optimization
• richer set of operators
• pre-trained models
• loading data, I/O
• etc.

Issues

• None of the mobile browsers are supported
• Firefox is rather slow and has unpredictable behavior – #3
  • please use Chrome for best experience
• Edge looks visually off, but works OK

Contribution

Any form of help and feedback is welcome – including PRs, reporting bugs, suggesting ideas, shooting down existing ideas, creating new examples, promotion, testing, etc.

This project is under MIT license.

---

Thank you

Big thank you to these great projects and awesome people behind them:

• TensorFlow.js
• Ohm
• Monaco Editor
• FiraCode
• React
• RxJS
• etc.

This thing is stealing great ideas from:

• APL family – A, J, K, Q
• LISP family – ClojureScript
• ECMAScript
• QML
• JSON
• Jsonnet – Ren
• Haskell
• Smalltalk
• Moniel – prototype of L1
• Douglas Crockford
• Bret Victor
• etc.

l1's Issues

Observables in and out

Description

In Observable-based interpreter, every function takes a raw JS value and returns an Observable. This breaks composionality and it is a bad design choice.

This problem complicates/prevents implementation of following functions which were trivial in Promise-based interpreter:

• Flow (old implementation)
• Gradient (old implementation)

Solution

Obviously, every function should receive and return an Observable.

Consequences

• function can decide on how to combine the Observables (via operators)
• simple functions can be easily lifted to an Observable ones
• every value the interpreter sees is an Observable (object === Observable of object containing Observables? whoa, ble, haha, hmm :D)
• there can be (syntax) operators and functions for Observable operators
• proper Flow implementation (and others constructs)
• implementation of functions will be verbose (unless they are not automaticaly lifted)
• Mouse.x * 10 should work right away
• one can lost very easily in a higher-order Observables
• What will function application do?
  • combineLatest should get two second-order Observables (?)
  • therefore Interpreter should manipulate only second-order Observables (?)
• ...
• more to think about..

Too slow in Firefox

With ::Self it barely responses to typing.

Browser: Firefox 62.0b13

Fold (reduce)?

I just wanted to do {+/ ⍵*2}...

Reimplement prototype chain

Right now, L1 uses the same prototype chain mechanism as JS, i.e. objects in L1 are JS objects. This has a big limitation: only string and symbol keys for object values.

Solution: {} in L1 should be Map().

Using keys with any type would be highly beneficial, e.g. ability to have a function as a key would probably be the most twisted thing possible.

Another useful feature would be to have an ability to create an object from a specified prototype:

Proto: {
    prop: 23
}
Mu: {
    #proto: Proto
    prop2: prop + 24
}

Computed Property Key

In JS, you can use computed property name like this:

const key = "age"
const mu = {
    [key]: 47
}

In L1, you should be able to do this via string interpolation:

key: "age"
mu: {
    "$(key)": 47
}

This means that object should allow strings as prop keys, which is good since it will be closer to JSON.

---

Another way is using just parens, which looks really clean:

key: "age"
mu: {
    (key): 47
}

How to define computed symbol key props...?

key: "age"
mu: {
    (#(key)): 47  ; ugly
    (# key): 47  ; hmm
    #(key): 47 ; hmm2
    (Symbol key): 47 ; via function call (compatible with "(# key)")
}

String Interpolation

In JS, the string interpolation can be used only in the template literals:

const name = "World"
const greeting = `Hello ${name}`

In L1, the string interpolation is going to be available in normal strings:

name: "World"
greeting: "Hello $(name)"

Inside the parens can be any expression that should result in string. If the value of the expression is not a string, an error is produced.

Lists

Lists – ordered collection of any type

Lists will be implemented as arrays in JS. However, immutable.

Notation

• [expr, expr]
  • no, because tensors are more important right now
• (expr, expr)
  • weird, but nice
  • can be confusing
  • () will clash with None (doesn't have to)
    • can be empty list, but then these function calls can't be equivalent: fn!, fn()
      • there are functions that can take empty list as a "valid data": (Length () = 0) = (Length ! = !), (1,2,3) -> Map (v => v + 1), () -> Map fn, error: ! -> Map fn
  • (expr,) is single-element list
  • Python comma operator creates tuples
  • JS comma operator is a bad part, so can be altered

Pattern matching

Pattern matching is an extremely powerful construct and should not be left out of the language.

However, this seems like a really complex feature that will take a long time to get right. Binding free variables looks like the major pain-point implementation-wise.

---

Good start point may be just multiway conditional – this is using bools as object keys, which is prohibited, and other non-existing stuff, but lets see...

someBool: ...
result: someBool ? {
    (True): "Yup"
    (False): "Nope"
}

someTensor: ...
result: someTensor ? {
    ([]): "Empty tensor"
    (1): "One"
    ([1 2 3]): "Meh"
    #default: "Something else"
}

someObject: ...
result: someObject ? {
    ({}): "Empty object"
    ...
}

This seems useless...

---

Inspiration

Patterns in Swift
Pattern matching in ReasonML
Amber
Scala
Haxe

Notation

Should use existing constructs as much as possible. However, switch operator with a list of cases will probably be insufficient.

Idea 1

Idea 2

#repr

As __repr__ in Python, but not a string representation, but visual representation. This is already working for the builtin types (Tensors, Objects, etc.), but it should be accesible from the language.

Paired with HTML literals it would be really nice. Example pulled from the future:

A: {
    value: 23
    #repr: (
        <h1>{value}</h1>
    )
}

This has a very low priority.

Palette of functions

On Try APL there's a little cheat sheet with symbols. It would be nice to have something like that for L1 (::Self is too long).

Propagate first Error to the visualization

Displaying only last error in an expression is useless – first error should be propagated to the visualization.

If value in function application is Error, return that error without even trying to apply the function.

Callable Tensors

Callable Tensors, i.e. tensor as a function, can be a multi-purpose mechanism for:

1. Tensor indexing
2. Flow

Don't hide results on an error

This is distracting. Also, you usually want to see previous results while writing the code.

Online examples

To learn the language by examples.