pkpkpk / cljs-node-io

A ClojureScript IO Library for NodeJS

com.github.pkpkpk/cljs-node-io {:mvn/version "2.0.339"}

This is a port of clojure.java.io to clojurescript, in a way that makes sense for nodejs. The goal is to make the clojure programmer feel right at home, so most everything has the same signatures and semantics as their jvm counterparts. However many things necessarily work differently internally, and with some consequence. You can read about the differences here

Also included:

• reified files with same api as java
• slurp + spit
• async helpers, wrappers over child processes
• convenience functions to make your scripting and repl'ing experience more pleasant

---

Be synchronous in your repl sessions & scripts

(require '[cljs-node-io.core :as io :refer [slurp spit]])

(def data [{:foo 42} {:foo 43}])

(spit "data.edn"  data)

(= data (read-string (slurp "data.edn"))) ;=> true

Write your app asynchronously

;; write asynchronously using core.async
(go
  (let [[?err] (<! (io/aspit "data.edn" data :append true))]
    (if (some? ?err)
      (println "there was an error writing: " (.-message ?err))
      (println "successfully appended to 'data.edn'"))))

;; read asynchronously using core.async
(go
  (let [[err datastring] (<! (io/aslurp "data.edn"))]
    (if-not err
      (handle-data (read-string datastring))
      (handle-error err))))

Use files just as you would on JVM

(def file (io/file "data.edn"))

(io/delete-file file true)

(.exists file) ;=> false

(spit file data)

(.stats file) ;=> {:mtime #inst"2022-06-10T23:26:00.558-00:00", :isFIFO false, ...}

(def parent (.getParentFile file))

(filter #(.equals file %) (io/file-seq parent))

easy subprocesses

(require '[cljs-node-io.proc :as proc])

;; Warning! dont hot reload shell calls!

;; blocks the main thread
(proc/exec "pgrep java" {:encoding "utf8"})

;; does not block the thread
(let [[?err json] (<! (proc/aexec "rg -e foo --json" {:encoding "utf8"}))]
  (if (some? err)
    (handle-err ?err)
    (search-results (js/JSON.parse json))))

---

Async vs Sync naming conventions

In NodeJS, functions are async by default, and their synchronous versions have names with a Sync suffix. In cljs-node-io, functions are synchronous by default, and async versions have an a prefix. For example, cljs-node-io.core/slurp is synchronous (just as jvm), whereas cljs-node-io.core/aslurp runs asynchronously. a prefixed functions always return channels.

This convention simply saves you some thought cycles at the repl. Note that most of the time (scripting...) synchronous functions are fine and getting order guarantees from async code is not worth the hassle.

IO Operations & Error Handling

• all functions should throw at the call-site if given the wrong type.
• Sync IO ops will throw on IO exceptions. Error handling is up to the user
• in asynchronous functions, IO exceptions will be part of the channel yield. The convention here is to mimic nodeback style callbacks with channels yielding [?err] or [?err ?data] depending on the operation
• for successful ops, errors will be nil. This lets you destructure the result and branch on err
• note this is not transactional... some side effects may have occured despite an error

Predicates

• Sync predicates do not throw on op errors, they catch the error and return false
• Async predicates return chans that receive false on err. These channels only receive booleans.

---

Getting Order Guarantees From Async Code

read more here

(require '[cljs-node-io.fs :as fs])

(fs/touch "/tmp/hello")

;; BAD! maybe astat is run before arename
(def rc (fs/arename "/tmp/hello" "/tmp/world"))
(def sc (fs/astat "/tmp/world"))

(go
  (let [[?err] (<! rc)]
    (if (some? err)
      (handle-err err)
      (let [[?err stats] (<! sc)]
        (if (some? ?err)
           (handle-err ?err)
           (handle-stats stats))))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; GOOD!  chain the calls together
(go
  (let [[?err] (<! (fs/arename "/tmp/hello" "/tmp/world"))]
    (if (some? ?err)
      (handle-err ?err)
      (let [[?err stats] (<! (fs/astat "/tmp/world"))]
        (if (some? err)
           (handle-err err)
           (handle-stats stats)))))))

---

Differences from Clojure

• Node runs an asynchronous event loop & all IO is driven under the hood by libuv. Construction of streams involves creating a object within the js-vm and returning it to the user synchronously so that listeners may be attached. Calls to listeners are then scheduled using js/process.nextTick. This means you fundamentally cannot create a stream and consume it synchronously... you must instead create the stream and attach handlers to its emitted events.

  • clojure.java.io coerces everything into streams synchronously, and reads and writes from there. This strategy cannot work in node

• To preserve synchronous semantics, slurp for example uses memory consuming fs.readFileSync. This is fine for small files and repl sessions. If you need to read larger files, restructure your program to accommodate node streams. Luckily node streams mostly manage themselves.

• no reader + writer types, not really necessary

• no URL type, just goog.net.Uri

• javascript does not have a character type

• no java-style char/byte arrays, just nodejs buffers

cljs-node-io's Issues

Seems not to be advanced optimizations safe

It seems the node-externs does not support enough info for advanced optimizations.

renameTo fails with File-Object input

File Method "renameTo" fails with other input than string input (see

cljs-node-io/src/main/cljs_node_io/file.cljs

Line 176 in ad5a87c

(assert (string? dest) "destination must be a string")

). Java File Method "renameTo" accepts File as input (see https://docs.oracle.com/javase/7/docs/api/java/io/File.html#renameTo(java.io.File))

Incorrect opts structure has surprising results

Dear kind maintainer,
This may simply need none of your attention and can probably be closed, but just in case input validation strikes your fancy.

I some how decided that the following code was correct.

(io/aslurp "afile.txt" {:encoding ""})

It looked like all was well, my files were being read and pushed over the wire just as i expected. But I kept having errors reconstructing the files. After finally resorting to hexdumping the file I noticed lots of hex 0xEF 0xBF 0xBD in my output file. after running around in circles for quite some time i realized the only place this could possibly be happening is actually reading in the file. So I dug back into your docs and found that i was indeed using your library incorrectly. the correct answer was

(io/aslurp "afile.txt" :encoding "")

Now, of course, this was user error. But just a little sprinkling of input validation could save a soul such as mine.

Anyways thank you for putting this library together, despite my inability to read it has made my life quite a bit simpler.
❤️

Compilation warnings

I'm getting these warnings during compilation with more recent clojurescript (now on 1.9.908 but it started before that). This does not seem to break anything.

Aug 23, 2017 11:39:34 AM com.google.javascript.jscomp.LoggerErrorManager println
WARNING: .../cljsbuild/out/cljs_node_io/core.js:295: WARNING - Keywords and reserved words are not allowed as unquoted property names in older versions of JavaScript. If you are targeting newer versions of JavaScript, set the appropriate language_in option.
var or__7587__auto__ = cljs_node_io.core.file.call(null,f).delete();
                                                           ^

Aug 23, 2017 11:39:34 AM com.google.javascript.jscomp.LoggerErrorManager println
WARNING: .../cljsbuild/out/cljs_node_io/file.js:564: WARNING - Keywords and reserved words are not allowed as unquoted property names in older versions of JavaScript. If you are targeting newer versions of JavaScript, set the appropriate language_in option.
cljs_node_io.file.File.prototype.delete = (function (){
                                 ^

Aug 23, 2017 11:39:34 AM com.google.javascript.jscomp.LoggerErrorManager println
WARNING: .../cljsbuild/out/cljs_node_io/file.js:659: WARNING - Keywords and reserved words are not allowed as unquoted property names in older versions of JavaScript. If you are targeting newer versions of JavaScript, set the appropriate language_in option.
return this$.delete();
             ^

Aug 23, 2017 11:39:34 AM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 0 error(s), 3 warning(s)