enlivez's People
enlivez's Issues
Day #9
The proper clj/cljs split has begun. Updates work on a pseudo dom (that's going to be wrapped in reagent for a start).
Relational expressions
A relational expression is an expression. I use the word expression in contrast to statement (see https://en.wikipedia.org/wiki/Expression_(computer_science) for example). Expressions nest, statements don't.
In typical datalog, clauses don't nest. To me it's a loss in expressivity because it means that you must name a lot more of variables.
In Datalog a predicate has no direction: it has no output or input attributes, all attributes are full duplex so to say. However while a predicate may not technically have a direction, it generally has a semantical one: that's why by default the last argument is considered to be the return attribute.
Tables with dynamic columns
Day #3
Spent too much time on #6 to figure out that the simplest solution was:
- to normalize the query to DNF
- compute the composite key of each branch
- take the union of the keys
- it's ok to include a column (var) in the key even if it was not part of the initial
:find
.
Focused on this subproblem, I almost forgot why it is important: a template query may retrieve data only useful for child templates (often queryless) and we don’t want to trigger a redraw of a component when only data for a su component has changed.
Implementing rules
Adding rules to EZ has two impacts: visible and not.
The user-visible impact is (defrule ...)
(or something in this spirit) offering a way to factor code (and the experience so far proves that it's way due).
The invisible impact is that there's currently a LOT of queries which are run at each change and these queries have common parts (they can be organized as a tree and that's something I've started doing and this tree follows the structure). Using rules (well idb predicates) would allow a more natural way to structure the sharing (and also to get shared computation).
This would also allow recursive widgets (trees yay!).
Day #6
Ok, so I have figured the 3 basic types of templates (I can think of at least one more but less essential):
with
It's for looping, it has one query, one child, an optional sort.
foreign
Foreign nodes are leaves of the template tree: that's where EZ stops, usually foreign nodes will render to attributes or text nodes, but they may return full blown DOM nodes (eg to include a React component).
Foreign nodes have no child, but vars they depend on and a function to render and update.
Foreign components have state.
fragments
Fragments are fill-in-the-blanks html. They have no query, they don't use vars on their own, they do have children (one for each blank).
Summary
type | with | foreign | fragment |
---|---|---|---|
query | yes | no | no |
vars | no | yes | no |
children | 1 | 0 | N |
lifting ors
Imagine a rule such as:
(defrule is-all-powerful [user]
(info user name group) (or (= "root" user) (= group "admin"))
I want to lift the or out the rule to go to plain datalog.
My first idea was to lift ors as rules but then in this case we would have the following rules
xxx(name, group) :- eq(user, "root")
xxx(name, group) :- eq(group, "admin")
which violates the constraint that all variables appearing in the head must be grounded.
Day #10
It renders!
Both rendering and derivations are not the real thing (resp. using reagent and sets difference) BUT updates are local while we have a global model.
Plan for tomorrow: event handlers.
Collapsing empty sections
Imagine you have a header followed by a list and that you don't want to display the whole section where there's no entry in the list.
(ez/for {:db/id list :list/name name :list/entries _}
[:div
[:h1 name]
[:ul
(ez/for {:db/id list :list/entries {:item/title title}}
[:li title])]])
is the best one can actually do.
In the absence of rules the query must be duplicated (and this can be a maintenance woe if some filtering is going on).
How could this be avoided syntactically?
Entity-like syntax
At some point when doing exploratory design, I played with entities. However they are too powerful, too dynamic: once they are passed to clojure code you can't know what they are going to be used for (and no, wrapping and tracking is not an option).
However the appeal was just to be able to write (:item/title item)
in the template, not being forced to destructure upfront.
Hence the following idea: make (:attr eid)
work in templates (and all variants).
Handlers and activations
I have a typical handler which is a relational expression
(ez/defhandler end-edition [selfx itemx]
[[:db/add selfx :editing false]
[:db/add itemx :title (:draft selfx)]])
The issue is that if it becomes a rule it needs to be passed an activation because its argument must be known before hand (because it's mostly doing interop/impure calls).
I'm starting to realize that these activations look suspiciously like supplementary predicates.
TODOs
- constant fragments in loops
- less awkward predicates
- sorting
- proper shadowing (lexical scope)
- inclusion
Deriving negation
You just swap added and retracted.
Day #1
Working on the working prototype. The goal is to get the lib in the hand of devs asap even if the implementation is utterly naive and inefficient.
Some realizations of the day: component identity, query maps in arguments and subscriptions.
Component identity
A component is fully identified by the arguments that were passed to it (if they change then the component is replaced, not mutated) and its mount point (relative to its parent).
However one must not panick at these words: when a component represents a todo-list item, its argument is the list item entity id which is stable. Changing the item title would not replace the component, just refresh it (or some subcomponents).
Query maps as arguments
When using query maps as a kind of destructuring (for example {[title status] ::item/attrs}
) in arguments there's the risk the resulting query may return 0 to many rows where we expect only one.
Too many rows: this can be prevented by analysis of the query and attributes cardinality.
For zero rows returned (eg because the entity as a missing field and no default has been provided), this has to be dealt with at runtime. Great, but how? Through a "default default value" (nil
being a candidate)? Warning? Exception?
Subscriptions
So in our model, the database is not available as a first class value (except in transaction functions).
One has to subscribe to a (rel-returning) query. If the query returns N columns then the subscribed handler receives a relation with N+1 columns: the last one is a boolean, true
for addition, false
for retractation.
I was uneasy with using d/listen!
to implement it because you have to take Javascript "thread model" in account to be sure that there's no lost transaction between the initial query (used to bootstrap the live-query) and the first delta. (There are 2 atoms: one for the listeners, one for the db...)
While it can be worked through, I opted for a simpler and more general solution that works on the JVM too. This solution is to have only one listener (created at db init), and all subscriptions are reified in the db itself!
Thus a live query kicks off the moment its subscription is transacted!
;; Datasource is only accessible through subscription
(def ^:private conn
(doto (d/create-conn {::child {:db/valueType :db.type/ref
:db/isComponent true
:db/cardinality :db.cardinality/many}})
(d/listen! ::meta-subscriber
(fn [{:keys [tx-data db-after]}]
(doseq [[eid q f] (d/q '[:find ?eid ?q ?f :where [?eid ::live-query ?q] [?eid ::handler ?f]] db-after)]
(f (d/q q db-after)))))))
(defn subscription
"Returns transaction.
Upon successful transaction, the "
"Subscription is immediate: upon subscription f receives a (positive only) delta representing the current state."
[q f]
[{::live-query q
::handler
(let [prev-rows (atom #{})]
(fn [rows]
(let [prev-rows @prev-rows
added (reduce disj rows prev-rows)
retracted (reduce disj prev-rows rows)
delta (-> #{}
(into (map #(conj % true)) added)
(into (map #(conj % false)) retracted))]
(when (not= #{} delta)
(f delta)))
(reset! prev-rows rows)))}])
Day #8
Small victory
=> (let [adom (atom nil)
root (fragment-component [:ul :_]
[[[1] (with-component
(fragment-component [:li :_ "is" :_]
[[[1] (terminal-component identity)]
[[3] (terminal-component #(if % "done" "yet to do"))]]))]])
root (root #(reset! adom %))]
(doto root
(ensure-path! [[0] [12345] [0] ["title"]])
(ensure-path! [[0] [12345]])
(ensure-path! [[0] [12345] [1] [false]]))
@adom)
[:ul ([:li "title" "is" "yet to do"])]
Why no public or ambient conn or db?
No db to enforce all reads to be declared (no ad hoc reads hidden in Clojure code) at the template level (but we can have non-dom components: IO and History API fro example).
No conn to enforce that the only way to alter state is through tx-data returning handlers.
tx function is the only place where random queries can occur.
Unstub derivatives
Currently derivatives are mocked by brute force diff
Day #14
States, as always. Now it's datascript that is buggy:
(d/q
'[:find ?a ?v ?s
:in $x
:where
[$x ?a]
[(vector ?a) ?v]
[(str ?a) ?s]]
[[1] [2]])
#{[1 [2] "1"] [2 [2] "2"]} ; should be #{[1 [1] "1"] [2 [2] "2"]}
aggregations
aggregations once implemented may suffer of the same query duplication issue as #36
Day #11
Event handlers are working.
Now, on to component state!
`:state` maps shouldn't be query maps
Query maps make for awkward state maps for two reasons:
1/ A query may return 0, 1 or n results but for a state entity we expect strictly one component
2/ Defaults are defaults, if an attribute value is retracted then the default is back on; this is a counter intuitive behavior to me.
State maps should be more like entity maps, they should only use (in values) symbols which are part of the component's key, so that the initialization state is stable.
This would also allow for many-valued attributes to be set too.
Day #7
Thinking out loud about what happens at runtime (todo list example).
- A new item is added to the database.
- It triggers the LQ for the
with
, this LQ emits a path[0] [] [12345] ["title" false]
:[0]
is the key in the root fragment (the one with the<ul>
) to identify its first and only child: thewith
. Then[]
is additionals values (always empty fro fragments) then a key anew[12345]
this time its the eid of the new item, it's the identifier of the iteration, and then additional values.- The thing that bugs me at the moment is that terminal (foreign) templates don't have queries and thus need support from the fragment and with to render: the with must pass its "additional values" to the fragment which must pass them to its terminals and implement logic of checking which values have changed!
- Let's imagine a world where we get 3 paths
[0] [12345]
,[0] [12345] [0] ["title"]
and[0] [12345] [1] [false]
then, in this world, we have full selectivity and all the diffing logic is removed from fragments etc. We also don't have the keys/other vals split
How many subscriptions?
(todo list example)
If each component has its own query then for a todo-list of N items we get at least N+1 subscriptions (1 for the list and N for each item).
Having 2 queries sounds better; is it even doable to have only 1 query?
Is `:state` needed at all?
Day #5
Todo: use keys (as per #11) in the fake derivatives
Recursive templates
Let's imagine displaying a tree.
(ez/deftemplate node [self]
(ez/for {:db/id self [name] :node/attrs}
[:div name
(ez/for [[self :node/children child]]
(node child))]))
Currently it fails as the system tries to get a tree of queries out of this and this tree ends with an infinite depth.
Obviously recursive templates may be dangerous in the same way ...
is dangerous in datomic's pull
. A recursion limit option on the template may be good.
Less awkward predicates
currently we have things like
(ez/for [[(pred expression)]] ...)
which is not great
Day #2
What is the runtime model of the UI? A hierarchy of components.
When a delta arrives we get a long row, which can be split into stages (see rules in #4) , each stage mapping to a component into the hierarchy.
If there's no component then it should be instantiated.
Let's imagine (syntax with #5):
[:ul
(with [[item :item/title]]
[:li (:item/title item) " is " (if (:item/done item) "done" "to do")])]
Here I count 4 templates:
- outer with
- inner with
- title
- done
Which would mean 3 queries
[[item :item/title]]
- inner with has no query and should never have one
[[item :item/title] [(get-else item :item/title nil) title]]
[[item :item/title] [(get-else item :item/done nil) done]]
The last two would need to be differentiated by adding an id (the mount point id for example): [[item :item/title] [(ground :mount_421) mnt] [(get-else item :item/done nil) done]]
Thus such code should work:
(defn add [{:as component :keys [n instantiate]} row]
(if (seq row)
(let [k (subvec row 0 n)]
(update-in component [:children k]
(fn [child]
(add (or child (instantiate k) (subvec row n))))))
component))
It's too functional/persistent for something which is linear.
Unique composite key on rows
Given a query how do I determine a subset of columns that are enough to uniquely identify a row?
Day #4
Keysets (cont & end)
Ok after admitting that I'd better go the DNF route to compute a keyset (set of columns that are a key on the result set) I realized that if the query had a cycle (does it occur often in practice?) my algorithm would fail to return a proper key.
As usual to deal with cycles, at some point I have to switch from a graph of nodes to a graph of SCCs. It works, in the absence of cycle the keyset is minimal, with cycles the keyset may not be minimal (there may be redundancy but it no case non-key vars would be included).
Birth of a protocol
Here we go (there's nothing yet about instantiation)
(defprotocol Template
(query [t]
"returns the query for this template")
(used-vars [t]
"vars used in its own body (not by nested templates)")
(children [t] "returns a map of ids (mount points) to templates"))
So, using the TODO example with 4 templates: the with
, the inner with
, the title and the status:
- only
with
returns a query:{[title done] :item/attrs}
whose keyvars are[?id421]
(the item eid);with
has no used vars on its own but it has one child, - inner
with
has no query, no used vars and two children, - each children has no query, one used var each and no child.
(I wonder if "inner with" is there to stay.)
Death of a protocol
A protocol where all methods have no args is not a protocol: it's a map!
Let's start with a simple case: the foreign template -- that is the template which calls clojure code.
(defn foreign-template [vars f]
{:vars vars
:instance f})
The idea is to call f
with values bound to vars
.
Now, to with
:
(defn with-template [q body]
{:query q
:children [body]})
So q
is executed and for each returned row, children are instantiated. Each instance is identified by the key (derived from the query).
Unstub DOM
True dom, remove reagent/react
Better derivatives
According to #6, result sets are not real sets they have a. Key. However this information is not leveraged by derivatives: a change is conveyed as a retraction and an addition of tuples sharing the same key. Reporting just the addition is enough because key unicity implies that this addition is an upsetting.
Likewise retraction just need to specify the key.
This should simplify derivatives.
Main algorithm
Reminder: template is to component what class is to object.
- Retrieve all queries from the template hierarchy
- Instantiate the root component (technical root component no matching DOM)
- Run queries derivatives, for each upsert row, traverse the component tree until matching (key-wise) component is found
- If found, update the component
- If not found, instantiate all missing components (using children template information from the last traversed component)
- for each delta row, traverse the component tree until matching (key-wise) component is found and remove it (retract state entity too)
Day #12 and #13
Working on state, root state works, nested states have conflicts.
I stumbled on a couple of CLJS bugs: https://clojure.atlassian.net/browse/CLJS-3092
and https://clojure.atlassian.net/browse/CLJS-3093
transact! in subscribers
subscribers being run on the listener it means they can recursively trigger themselves.
transactions should be postponed until the end of the meta listener. (.setTimeout 0)
may well be enough.
Allow expressions in sort key
as a way to handle changing sort orders
should it be done query side or component side? (eg like a template) query side seems better
how to handle asc/desc for non-numeric values?
Days #15 #16 #17
inclusion and sort!
Clean for-component
for-component
is a mess of reifies now that sort is implemented
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.