Making entities transactable about datalevin

[den1k]

@huahaiy adapted Entity to allow assoc, dissoc, add and retract. This works by adding a immutable "stage" to Entity that does nothing until the Entity is transacted. For example:

(-> (entity @conn [:user/handle "ava"])
    (assoc :user/age 42))

; => {:db/id 1, :<STAGED> #:user{:age [{:op :assoc} 42]}}

I like this interface because it is fully backwards compatible. Entity still works read-only but can optionally act as an immutable stage until transacted.
A lot of the code has been copied from datalevin.impl.entity to match the new Entity type. However, there are maybe only 50 lines of code that are actually new.

Here's a thorough API example:

(def db-path "data/lab/entity-db")

(def schema
  {:user/handle    #:db {:valueType :db.type/string
                         :unique    :db.unique/identity}
   :user/address   #:db{:valueType   :db.type/ref
                        :cardinality :db.cardinality/one}
   :address/street #:db{:valueType :db.type/string}
   :user/friends   #:db{:valueType   :db.type/ref
                        :cardinality :db.cardinality/many}})

(def conn
  (d/create-conn db-path schema))

(transact! conn [{:user/handle  "ava"
                  :user/friends [{:user/handle "fred"}
                                 {:user/handle "jane"}]}])

;; *** Simple example
(let [ava-with-age (-> (entity @conn [:user/handle "ava"])
                       (assoc :user/age 42))]
  #spy/c (entities->txs [ava-with-age])
  ;; => [[:db/add 1 :user/age 42]]
  (transact! conn [ava-with-age])
  )


;; *** Nested entities must be transacted separately
(let [{:keys [user/friends] :as ava}
      (update (entity @conn [:user/handle "ava"]) :user/age inc)
      fred   (some
               #(when (= (:user/handle %) "fred") %)
               friends)
      bestie (assoc fred :bestie? true)]
  #spy/c (entities->txs [ava bestie])
  ;; => [[:db/add 1 :user/age 43] [:db/add 2 :bestie? true]]
  (transact! conn [ava bestie]))

;; *** `add` and `retract` are directly defined on entity
;; they differ from assoc/dissoc in that they do not overwrite
;; the attr's values
(let [ava  (entity @conn [:user/handle "ava"])
      fred (some
             #(when (= (:user/handle %) "fred") %)
             (:user/friends ava))]
  #spy/c (entities->txs [(retract ava :user/friends fred)])
  ;; => [[:db/retract 1 :user/friends 2]]
  (transact! conn [(retract ava :user/friends fred)])
  )

All the code is here: https://github.com/den1k/stuffs/blob/main/src/stuffs/datalevin/entity.clj
Thoughts?

[huahaiy]

I am all for programmer convenience (that's the whole point of this project), so I think this could be a nice addition. Would you mind sending a PR with some tests as well. Thanks a lot!

[den1k]

This also possibly unlocks use of libraries like specter or meander to update entities.

[huahaiy]

Sounds like a good idea. Could you elaborate a bit more on how specter or meander could be used to update entities if the change is made?

[den1k]

Could you elaborate a bit more on how specter or meander could be used to update entities if the change is made?

Here's a basic example with specter based on the entities above:

(use 'com.rpl.specter)

(def ent-as-map-recursive
  {:db/id        1
   :user/email   "[email protected]"
   :user/friends #{{:db/id      2
                    :user/email "[email protected]"} {:db/id 3}}})

(setval [:user/friends ALL :user/email (pred #(= % "[email protected]"))]
        "[email protected]"
        ent-as-map-recursive)

; =>
{:db/id 1,
 :user/email "[email protected]",
 :user/friends #{{:db/id 2, :user/email "[email protected]"} {:db/id 3, :user/email nil}}}


(setval [:user/friends ALL :user/email (pred #(= % "[email protected]"))]
        "[email protected]"
        (d/entity @conn 1)))
; => 
; Execution error (UnsupportedOperationException) at datalevin.impl.entity.Entity/assoc (entity.cljc:47).
; null

[huahaiy]

I wonder how to deal with deeply nested, or even recursively nested entities?

[den1k]

Yes, I started pondering that as well. I actually think that turning the Entity into a map is a bad idea. But we could possible extend the Entity type to hold a change-set of updates to the entity similar to the cache of touched.

The key is that the, sic, key is known, e.g. for
(update (d/entity @conn 1) :user/email str/upper-case) we know that only the value for :user/email changed, so we can translate that into [:db/add 1 :user/email "[email protected]] or {:db/id 1 :user/email "[email protected]} for the transaction, while ignoring unchanged keys and values.

This would be similar for nested values. Non-updated entities are treated as eids and updated ones contain a minimal change-set derived from assoc/update calls.