magnars / dash.el Goto Github PK

I've recently been playing around with implementing lazy streams in Elisp (https://github.com/shosti/smash.el), and it occurred to me that it might be possible to integrate laziness into dash.el instead of re-implementing lazy versions of everything. Is that something you'd be interested in pursuing? (I'm not yet sure of their performance or general usefulness for Emacs programming, but they allow for some neat things).

I think it might even be possible without breaking backwards-compatibility or requiring Emacs 24. Basically, you'd need a macro that would create two versions of each function, one that outputs a stream and one that outputs a list (maybe postfix lazy versions with a symbol, i.e. -take$ or something). The lazy version would only be generated in Emacs 24. In the macro, you'd also coerce the input to a list or stream (for the eager and lazy versions, respectively). smash.el already handles inputs that are either streams or lists, through some basic type-tagging.

This would probably be at odds with the approach described in #43, but it would have the advantage of being extensible to user-defined data structures--you'd just need a mechanism to "install" a coercion function for the data structure (similar to the seq abstraction in Clojure). Personally, I'm a fan of Clojure's "anything goes in, but seqs come out" approach to generic list functions, and I think guessing the desired output type without a static type system is kind of problematic. But I could also see laziness being outside the scope of dash.el (I don't really have any real need for lazy streams, and was just implementing them for fun). Let me know what you think.

ert.el is currently in the main project path. This will shadow the emacs' default ert.el if the project directory is added to the load-path. This is, e.g., a problem for el-get integration. See dimitri/el-get#974 (comment)

This could be fixed by moving ert.el into a separate subdirectory similar to what you did in js2-refactor.el (where it is in util/)

There's the group/groupby from Data.List, which is pretty useful. It's not yet present in dash and it generalizes -partition-by. The docs from hackage:

The group function takes a list and returns a list of lists such that the concatenation of the result is equal to the argument. Moreover, each sublist in the result contains only equal elements. For example,

 group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]

It is a special case of groupBy, which allows the programmer to supply their own equality test.

There already is -group-by in dash but with different semantics. I would advice to alias the function and deprecate the old name, then add the proposed functions with that name (after some time, so people can migrate their code).

The simple group could be omitted since it's just groupBy (==). -partition-by from dash can be implemented as (haskell code)

partitionBy f = groupBy ((==) `on` f)

Speaking about function names, I would strongly push towards regularizing the names of functions based on the haskell API. There isn't any API in elisp, so we can name it whatever, and having the same API as haskell would be benefitial for haskell programmers (and others from languages that inspired their APIs by haskell, which is a lot), while not being any drawback for elisp programmers who would need to learn some API anyway.

For example, -separate in haskell is partition, while -partition has completely differnent semantics in dash. -split-with is called span (span the predicate over longest prefix). The current -partition-* could probably be called more precisely too, something to do with sublists.

Also the usage of -with and -by is inconsistent in dash. Sometimes they take functions (a -> b) and sometimes predicates (a -> a -> Bool). Suffix -by should always take predicates or comparators and -with unary functions.

I've already discussed this a little bit with Magnar but I need to clear my thoughts a bit.

So what I'm proposing we do is an API for alists, plists and hashtables (I'll call these "maps" later). The rationale is simple: these maps, especially the first two are used ubiquitously all through emacs, yet they lack a pleasant and consistent API. One example for all, assoc key alist but plist-get plist key---who the heck should remember all that! The naming is atrocious, the calling conventions random, the functionality... not very developed.

Dash has proved itself extremely useful and successful API for lists, that's one reason I want this to be "under the same label"---it will get better recognition and the more people use it the better, we will fix bugs and add more functionality much faster. It would probably make sense to put it in a different repo, because the documentation would get reaaaaaally long otherwise.

I would only focus on emacs 24. I feel that 23 is too old to bother with, and lexical scope is sexy. 24.4. is coming soon and the next one is probably going to be 25... hopefully Debian would catch up. However, if you feel we should also support e23, I'm not hardcore opposed against it, it would only make the code a bit more messy.

There are couple existing solutions like ht.el by @Wilfred and emacs-kv by @nicferrier, and while good on their own, they suffer the same weaknesses---the API is mostly contingent, solving problems the author felt most pressing. The advantage of one unified API is clear---user needs to learn only one system to manipulate all of these data structures. I'd be very happy if the above people contributed to this library, as well as other dash contributors.

I have a temporary repository https://github.com/Fuco1/petulant-dangerzone with a couple stubs and drafts, I'll update it sometime in the near future to reflect all the discussion below. But I'd like the real thing be under Magnar's account, simply because he has billion followers and that way we can spread this much faster (talk about parasitism!)

Here's the basic overview of what I have in mind.

Distribution

I'd distribute this as three separate packages dash-pl, dash-al and dash-ht/hash, but they should all reside in one repo, so the updates can be synchronized better. As you'd see later, the documentation for all three will also be the same, so people using any of these can come to the same place to read the specs. Finally, need of one doesn't automatically imply need for the other, so it's also a bit more polite not to dump billion things on the user.

Prefix

We will have a single consistent API available for all three, only differentiated by a prefix. One issue I'm not so sure about is whether we should also keep the dash prefix. On one hand, it's a "trademark" and easily recognizable sign of the dash & friends, but on the other many people dislike it, some to the extend of simply refusing to use dash because "it has ugly names". The prefixes could be al, pl and ht if we keep the - as well, in other case it'd need to be something longer, but we can't use alist, plist or hash because that's already used. Personally, I'd favor the - solution, because it solves the namespace problem efficiently and I doubt any other package would ever use it. So it's like a dedicated namespace for us.

Basic calling conventions

The first argument will always be the map itself, so we could use the -> macro from dash to thread it through expressions. The key will be the last argument for a simple reason: we want to allow nested operations, so the last argument will actually be of &rest type allowing more than one key, it will then do the lookup recursively. The other alternative was having the functions take key or '(key1 key2 ...), but I find this really user-unfriendly---the enumeration simply feels much more natural. The value, if any, will come before the key. This is a little bit unusual, but I feel well justified by the above. Plus, we can read it as "insert/put value at key" which feels more natural than "at key insert/put value" or some other readings. If there are any more arguments, they will come inbetween but in fixed order.

Naming

All the names will follow couple conventions, so that we get a consistent overall design. Generalized functions will be designated by various suffixes. I'll demonstrate it on function insert. By default, it would take a map, a value and a key, and update or insert the value at key. The keys would be compared by equal and the value would simply be inserted.

We can generalize it to allow other equivalences, not just equal. If the function takes an equivalence relation on keys, it will have a suffix -by. Therefore, insert-by would take a map, an equivalence, a value and a key.

Further, we could make the new value in the map as a combination of the supplied value and the old value. For example, how many times you simply wanted to bump a counter 1+ in a p/alist? I know I do that a lot. So we could have a -with variant that would take a a -> a -> a function, combining the supplied and old value to give a new one. For example, to bump a counter by 1, you would call it with function + and value 1. Simple insert can be realized with function (-const value).

And finally, we may want to combine not only the values but also the key. The suffix for that would be -withkey, which adds key as the first argument to the above function, making it k -> a -> a -> a.

The -by suffix can be also reasonably used on retrieval. Further we might want to get the value not by a key, but a general predicate. For example, in an alist like auto-mode-alist, we might want to get the first value where key regexp-matches the alist key. I propose -at suffix for this, reading "get value at key which matches predicate". The predicate would be of type k -> Bool.

These prefixes can be combined to give variety of more or less generalized functions, but always in same order: -with[key], -by/-at.

Functions which serve as predicates will end with -p as is the standard elisp style and will also be aliased to a version ending in ? as in schemes.

Arguments

Arguments would follow a simple naming scheme

• the map will be called map, be it alist, plist or hash
• the key will be called key &rest keys,
• the value will be called value
• the argument required by -with[key] will be fun
• the argument required by -by will be equiv
• the argument required by -at will be pred
• the argument implementing ordering on keys will be called comp

This allows us to have a single documentation for all three (and possibly other structures if someone cares to implement them). In rare case where some operation might not be possible for a certain DS, we would simply add a note somewhere informing the user about this, e.g. "This function is not available for hash tables".

Note that equiv should always implement equivalence relation and comp should always implement a total ordering, returning non-nil when keys are equal for equiv and when first is \leq to the second for comp.

Anaphora

We can support anaphoras in the same way dash does, using it and other for the arguments. I have not yet decided what would be an anaphora for ternary functions (like the one used in -withkey).

Destructive variants

I think it makes sense to have both non-destructive and destructive versions. Destructive versions can simply always be marked by suffix ! as is common in scheme and other lisps. But from the start, I'd focus on implementing "pure" versions (which means we still allow shared sub-structure, but all the necessary conses will be re-created). Destructive versions would probably provide a bit of a speed up and sometimes make the operations more natural, e.g. there would be no need to reassign the result to the variable: (setq map (-operation map ...))

Conclusion

I've learned a lot about APIs by contributing to dash, mostly that it's damn difficult to design. So I don't blame emacs devs for the mess, but I feel after 30 years it's time for something new. That said, the potentiality to get this to core is basically none, because "we already have cl" or similar argument. Meh. I don't care about core anymore anyway.

Thoughts?

Well. First of all the pred there isn't a predicate at all. It is a function that transforms a value into another value. What's more confusing is that it must be of type a -> Int for the function to work properly. This is all very limiting and confusing.

The -by there should indicate you give it a comparison function to compare two values in the list that returns t or nil depending on the result fo comparison (like the callback for -sort)

If you want to have an "order by known type" function it should take a transfomation function into some ordering. Since elisp doen't have an ordering type, it could be integers. This should be called -max-after (a transformation) or something like that and could be specified as:

(nth (-index-of (apply '-max (-map transformation list))) list) ;; note that -index-of doesn't exist yet

But it's not very often you can do a sensible transformation from whatever into integers, so the following solution is even better (and nicely reuses the available tools) and is much more general.

In Haskell there's a handy combinator on with which you can do something like

compare `on` head

this would return a comprison function that would order lists by their car. (compare is a polymorphic function a -> a -> Ordering). Or to order list of lists by length you'd do

compare `on` length

The interface in elisp would look something like

(-max-by (on '< 'length) '((1 2) (1 2 3) (1) ()))

so the first argument of on is a comparison function (returning t or nil) that works ON the results of the second function (and that is applied to the elements of the list).

N.B. You could then use the on combinator in -sort as well as other possible functions that would need a comparison function

Thoughts?

a remove element would be nice 😄 (synonym with delq, which I always forget the name of)

cons* is documented here: http://www.gnu.org/software/mit-scheme/documentation/mit-scheme-ref/Construction-of-Lists.html

it's a very useful form (in emacs 'cl package it's list*).

It constructs a list so that the last cdr is the last argument:

(-* 1 2 3) => (1 2 . 3)

(-* 1 2 (cons 3 nil)) => (1 2 3)

(-* 1 2 (cons 3 4)) => (1 2 3 . 4)

Would you be interested in an implementation of such a function?

I came up with this the other day:

(defmacro* let-while ((var expression) &rest body)
  "A simple binding loop.

VAR is bound to EXPRESSION repeatedly until `nil'.

BODY is evaluated each time."
  (declare
   (debug (sexp sexp &rest form))
   (indent 1))
  (let ((expression-proc (make-symbol "exprp")))
    `(let ((,expression-proc (lambda () ,expression)))
       (let ((,var (funcall ,expression-proc)))
         (while ,var
           (progn ,@body)
           (setq ,var (funcall ,expression-proc)))))))

(defun let-while-test ()
  (catch :io
    (let ((lines '("line 1" "line 2")))
      (flet ((get-line ()
               (or
                (pop lines)
                (throw :io :eof))))
        (let-while (line (get-line))
          (message "the line is: %s" line))))))

It's very useful for using with iterators. I'm not sure you want to go down the iterator route but this might still be useful.

It would be nice if dash had an info manual. Perhaps this could be achieved by:

• pandoc
• rewriting the skeleton in org, writing a new org generator and doing org -> info
• using some other format
• ???

If someone feels like implementing this, please :) Otherwise I'm just adding this here so we won't forget.

By docs I mean the README.md file. It is already clickable from within emacs's help system.

You know I'm fond of the current way ->/->> are indented, but I feel that in the interest of compatibility (and the principle of least surprise) we should probably use the same indentation settings in dash.el as in clojure-mode.

See #67.

While lists are the most popular portion of Emacs Lisp's sequential API we still have to consider other sequential structures like vectors and strings. Without support for them dash cannot replace the sequential API provided by cl-lib in all possible scenarios and that's a shame, since dash's API is so much better.

We've discussed this already at Twitter and I know that at least a few people like @lunaryorn and @joelmccracken support my suggestion. Ideally when operating on a string/vector the functions would return a string or vector, but even a list would do (I assume this would be much easier to implement).

Here is the debug info

Debugger entered--Lisp error: (file-error "Cannot open load file" "no such file or directory" "dash-functional")
require(dash-functional)
eval-buffer(#<buffer load-319253> nil "/Users/jeremybi/.emacs.d/elpa/dash-20130816.59/dash.el" nil t) ; Reading at buffer position 30175
load-with-code-conversion("/Users/jeremybi/.emacs.d/elpa/dash-20130816.59/dash.el" "/Users/jeremybi/.emacs.d/elpa/dash-20130816.59/dash.el" nil t)
require(dash)
eval-buffer(#<buffer load-634447> nil "/Users/jeremybi/.emacs.d/core/prelude-core.el" nil t) ; Reading at buffer position 1179
load-with-code-conversion("/Users/jeremybi/.emacs.d/core/prelude-core.el" "/Users/jeremybi/.emacs.d/core/prelude-core.el" nil t)
require(prelude-core)
eval-buffer(#<buffer load> nil "/Users/jeremybi/.emacs.d/init.el" nil t) ; Reading at buffer position 3606
load-with-code-conversion("/Users/jeremybi/.emacs.d/init.el" "/Users/jeremybi/.emacs.d/init.el" t t)
load("/Users/jeremybi/.emacs.d/init" t t)
#[0 "^H\205\262^@ \306=\203^Q^@\307^H\310Q\202;^@ \311=\204^^^@\307^H\312Q\202;^@\313\307\314\315#\203*^@\316\202;^@\313\307\314\317#\203:^@\320\nB^R\3$
command-line()
normal-top-level()

There's -intersection and -difference, I see no reason why -union should not be added.

Here's a snippet for that (I don't think it's even worth forking)

(defun -union (list1 list2)
  (let ((result (nreverse list1)))
    (--each list2 (when (not (-contains? result it)) (!cons it result)))
    (nreverse result)))

It works very similarly to -distinct, only initialize the result with reverse of list1. The resulting list has proper order, that is list1 followed by elements of list2 that were not on list1.

The run-time isn't the best possible, but I think it's apropriate for a basic implementation: O(m*n). But then, most of the rest are O(n^2)...

Anyhow, this library is AWESOME! Ever since I've found it I use it everywhere (it's like having haskell in lisp, which is so cool :)) Plus, I really like the name :D

Edit:
Here are the docs and some examples.

"Return a new list containing the elements of LIST1 and elements of LIST2 that were not present in LIST1. The test for equality is done with `equal', or with `-compare-fn' if that's non-nil."

(-union '(1 2 3) '(3 4 5)) ;; => '(1 2 3 4 5)
(-union '(1 2 3 4) '()) ;; => '(1 2 3 4)
(-union '(1 1 2 2) '(3 2 1)) ;; => '(1 1 2 2 3)

However, I'm not sure if nreverse produce "new list". If not, the initalization should be done with --each to generate a copy first (which will also conveniently reverse it)

• -split-at, -split-with and -separate should return (a . b) instead of a list. It makes accessing the elements needlessly obtrusive (cadr instead of cdr for the 2nd item). These functions will most likely never return tripples so the list return value has little meaning there.
• -partition-by should be renamed to -partition-with. the -by prefix should be kept for predicative use. There can be, in fact, a new -partition-by that would compare the element to the previous via some equivalence (now this uses equal by default). So it would take a comparator or predicate. This is related to #49, a -partition-by with a predicate is basically groupBy from haskell (in the thread is also the implementation of proposed -partition-with)
• same goes to -partition-by-header, in fact, it can also be implemented in terms of the above function (I think).
• -group-by could also be made more powerful. As I see it, it basically computes "quotient sets" of equivalence relations, but the one supplied is implicit by equal here. An actual -quotient-by would be also cool (it would work much like -partition-with but without respecting the order, that is it would truly merge all equivalent partitions). So, -group-by should be renamed -group-with, and -group-by should take a predicate to compare the "keys" too. -quotient-by is the same but with dropping the car of the results (which means it doesn't need explicit transformation argument either)

Note that all the "implicit split functions" can be implemented in terms of the predicative versions, either by -on combinator or by explicitely stating the transformation for keys.

The new api:

• -split-at returns ((-take n list) . (-drop n list))
• -split-with is renamed to -split-by and returns ((-take-while pred list) . (-drop-while pred list)). Alternative name is -split-while.
• -separate returns ((-filter pred list) . (-remove pred list))
• -partition-by renamed to -partition-with, takes (fn list)
• -partition-by takes (comp list), splits each time the successive elements aren't equal by comp. Note that -partition-with is -partition-by (-on 'equal fn)
• -partition-by-header, same as above.
• -group-by renamed to -group-with, takes (fn list), returns alist ((result1 x11 x12 ...) (result2 x21 y22 ...)) where xij maps to result_i. This function makes a little bit of sense also with the name -group-by, but I think for consistency it's better if all the functions where there is a transformation of input to something via (a -> b) function should be called the same, that is with suffix -with. The argument for -by suffix is that it partitions by equivalence on the result of the transformation, so two elements are equal if they map to the same thing => this is the kernel of the map. But the fact that it makes it into alist is "dirty" form algebraic POV :P
• -quotient-by takes (comp list) and calculates the quotient set of list by eqv induced by comp. The union of all the partitions gives back original list (not necessarily in same order). There can be a variant that makes all the elements in the subsets unique as well. Note that -group-with is same as (--map (cons (fn (car it)) it) (-quotient-by (-on 'equal fn) list))

Ideas, comments?

Save the following code as installdash.el:

(require 'package)

(defun main ()
  (setq package-user-dir (expand-file-name "dash-elpa"))
  (add-to-list 'package-archives
               '("melpa" . "http://melpa.milkbox.net/packages/"))
  (package-initialize)
  (package-refresh-contents)
  (package-install 'dash))

Running it with emacs -Q --batch -l installdash.el -f main produces the following output:

Contacting host: melpa.milkbox.net:80
Saving file /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/archives/melpa/archive-contents...
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/archives/melpa/archive-contents
Contacting host: elpa.gnu.org:80
Saving file /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/archives/gnu/archive-contents...
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/archives/gnu/archive-contents
Contacting host: melpa.milkbox.net:80
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash.el
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-autoloads.el
Making version-control local to dash-autoloads.el while let-bound!
Generating autoloads for dash.el...
Generating autoloads for dash.el...done
Saving file /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-autoloads.el...
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-autoloads.el
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-pkg.el
Checking /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559...
Compiling /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-autoloads.el...
Compiling /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-pkg.el...
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash-pkg.elc
Compiling /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash.el...

In -grade-up:
dash.el:795:8:Warning: -map called with 1 argument, but requires 2

In -grade-down:
dash.el:805:8:Warning: -map called with 1 argument, but requires 2

In -sort:
dash.el:968:8:Warning: -sort being defined to take 2 args, but was previously
    called with 1-2
Wrote /var/folders/t4/wjkpdwzx26x9vcf1pql981gm0000gn/T/installdash549346AZ/dash-20140108.559/dash.elc
Done (Total of 2 files compiled, 1 skipped)

Please note the byte compiler warnings.

Is this a bug in dash? Or am I doing wrong? Or can I just ignore these?

In Python there's a function cycle that infinitely repeats a list.

>>> from itertools import cycle, islice
>>> def take(n, iterable):
    return list(islice(iterable, n))
... ... 
>>> take(4, cycle([1,2,3]))
[1, 2, 3, 1]

I propose to implement cycle function. If Emacs doesn't support lazyness and infinite lists, cycle could accept a length argument. This would allow to solve problems like zip up two list of different lengths in elisp.

I'm pretty surprised this isn't here already! :D I just had a need for this so here's the code:

(defun -zip (list1 list2 &optional init)
  "Zip the two lists together.  Return the list where elements
are cons pairs with car being element from LIST1 and cdr being
element from LIST2.  The length of the returned list is the
length of the shorter one.

Optionally, LIST2 can be a function. Then the values that it
generates are taken instead.  The function should take one
argument which is the last generated value.  INIT is the initial
value for this function.  The first value inserted into the
zipped list is INIT, the second is (list2 INIT),
third (list2 (list2 INIT)) etc."
(let ((r nil))
    (if (not (functionp list2))
        (while (and list1 list2)
          (!cons (cons (car list1) (car list2)) r)
          (!cdr list1)
          (!cdr list2))
      (let ((v init))
        (while list1
          (!cons (cons (car list1) v) r)
          (setq v (funcall list2 v))
          (!cdr list1))))
    (nreverse r)))

Examples:

(-zip '(1 2 3) '(a b c d)) ;; => ((1 . a) (2 . b) (3 . c))
(-zip '((1 x) (2 y) (3 z)) '(a b)) ;; => (((1 x) . a) ((2 y) . b))
;; index the list
(-zip '(a b c d e) #'1+ 1) ;; => ((a . 1) (b . 2) (c . 3) (d . 4) (e . 5))

The last example is in fact so cool that it maybe even deserves its own function. Up to you.

Installing the latest dash via ELPA fails during compilation with this final line of,

dash.el:952:1:Error: Cannot open load file: dash-functional

Looking in the package directory under .emacs.d/elpa/dash-20130816.59 it seems like that file, dash-functional.el, is missing from the package now?

My Compile Log returns:

Entering directory `/home/marcin/.local/share/idemacs/elpa/dash-20140811.523/'

In -flatten-n:
dash.el:303:43:Warning: reference to free variable `it'

In -table:
dash.el:951:25:Warning: value returned from (car v) is unused

Is there any chance to fix and silence it?

dash is mirrored on the Emacsmirror, which is a large up-to-date collection of Emacs packages.

As the maintainer of the mirror I am trying to resolve feature conflicts that result from one package bundling libraries from another package. I suspect in most cases these libraries were included so that users would not have to find, download and install each dependency manually.

Unfortunately bundling also has negative side-effects: if the bundled libraries are also installed separately, then it is undefined which version actually gets loaded when the respective feature is required.

Initially that isn't a big problem but in many cases upstream changes are not included or only after a long delay. This can be very confusing for users who are not aware that some of the installed packages bundle libraries which are also installed separately. In other cases bugs are fixed in the bundled versions but the fixes are never submitted to upstream.

Also now that Emacs contains the package.el package manager there is a better way to not require users to manually deal with dependencies: add the package (and when that hasn't been done yet the dependencies) to the Melpa package repository. If make is required to install your make you might want to add it to the el-get (another popular package manager) package repository instead.

Alternatively if you want to keep bundling these libraries please move them to a directory only containing bundled libraries and add the file ".nosearch" to that directory. You can then load the library using something like this:

(or (require 'bundled nil t)
    (let ((load-path
           (cons (expand-file-name "fallback-libs"
                                   (or load-file-name buffer-file-name)
                                   load-path))))
      (require 'bundled)))

Of course if your version differs from the upstream version this might not be enough in which case you should make an effort to get your changes merged upstream.

dash bundles at least the following libraries:

• ert

Best regards,
Jonas

Here is a interesting problem: I was running the test suite on my Mac, and it hung on the test of -fixfn. I then ran the tests in a Linux virtual machine with no problem.

Closer inspection reveals that there are some subtle platform-specific (or Emacs minor-version-specific?) differences in the floating-point math in the cos function. While the fixpoint of cos can be verified on both systems

(= (cos 0.7390851332151607) 0.7390851332151607) ;; => t

On a Mac (tested on GNU Emacs 24.4.1 x86_64-apple-darwin13.4.0, NS apple-appkit-1265.21), -fixfn never converges on a solution because cos oscillates between 0.739…8 and 0.739…5.

(cos (cos 0.7390851332151608)) ;; => 0.7390851332151608

(cos (cos (cos 0.7390851332151608))) ;; => 0.7390851332151605

(cos (cos (cos (cos 0.7390851332151608)))) ;; => 0.7390851332151608

On Linux (tested on GNU Emacs 24.4.91.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10)), however, cos does converge to the value specified in the test suite.

(cos (cos 0.7390851332151608)) ;; => 0.7390851332151607

(cos (cos (cos 0.7390851332151608))) ;; => 0.7390851332151607

(cos (cos (cos (cos 0.7390851332151608)))) ;; => 0.7390851332151607

As a temporary solution, I will submit a pull request in a moment for a change to run-tests.sh that will allow the test to be skipped.

To solve the larger issue, what do you think of the following solution, which would allow the caller of -fixfn to account for the specific nature of the function being analyzed?

• Modify -fixfn to take two addtional optional arguments
• an equality test function, which defaults to equal, but which could
  be a fuzzy comparison of floats.
• an iteration limit function, which defaults to a simple counter
  with a default max value. The caller could alter the max value or
  specify some more sophisticated convergence test.
• Modify the tests for -fixfn so that they use a fuzzy comparsion
  for floating point numbers such as the one given in the Emacs manual
  (with a narrower tolerance)

(defvar fuzz-factor 1.0e-15)
(defun approx-equal (x y)
  (or (= x y)
      (< (/ (abs (- x y))
        (max (abs x) (abs y)))
     fuzz-factor)))

If you agree on this approach, I can work on this futher and submit a pull request.

This may be a Melpa issue, but I thought I would flag it nonetheless (it does appear to be a dash only issue).

Case in point, http://melpa.milkbox.net/packages/dash-20130408.1637.el will currently 404.

-any? is a predicate returning a boolean. -first returns me the element, not the value of the "predicate" applied to it. Is there anything better than (car (-keep 'pred list))? If not, how would I go about implementing it?

Most of the time we write the destructure like this:

(&plist :key key :otherkey otherkey)

where the variable to which we destructure is the same as they key sans the :. I'm proposing this: when * is appended to the selector (so it's e.g. &plist*) the name of the binding variable is derived from the key in the following "smart way":

• a key :foo is converted into foo variable
• a key foo is converted into foo variable
• a key "foo" is converted into foo variable

This covers pretty much 99% of use-cases of mapping structures.

Ideas? Instead of * we might use a different symbol, but stars often means some variation in lisp so I think it fits well.

This code uses lexical-binding, which is not available in Emacs < 24.

As a result, if it's included in the dash package, then the entire package should depend on (emacs "24"), which is probably undesirable since many other Emacs 23-compatible packages would then transitively depend on Emacs 24.

Possible resolutions:

1. Use lexical-let instead if possible and drop reliance on lexical-binding
2. Distribute dash-functional as a separate package which depends on dash and Emacs 24

Hi, I've ran into something pretty strange.

I have a project called omnisharp-emacs that uses your library. I
recently found the -let forms (and friends) that allow destructuring
and like them a lot.

However, I discovered that while the code can be compiled and tested,
when installing from MELPA the destructuring forms I use give an error:
Invalid function: --mapcat

You can see the code here:
https://github.com/OmniSharp/omnisharp-emacs/blob/melpa-testing/omnisharp.el#L20

I see internally --mapcat is indeed used, but don't know why this
would be an error in my code. Indeed I found out --mapcat itself
compiles just fine with this method.

Since the problem is pretty contrived I'm not sure who to talk to -
maybe you could have an idea?

I set up a sample environment with a test script that you can try to
replicate the error:

mika@lusikka /tmp
  % git clone https://github.com/OmniSharp/omnisharp-emacs.git omnisharp-emacs-test                                                                                                                          
Cloning into 'omnisharp-emacs-test'...
remote: Counting objects: 2113, done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 2113 (delta 23), reused 0 (delta 0)
Receiving objects: 100% (2113/2113), 2.08 MiB | 906.00 KiB/s, done.
Resolving deltas: 100% (1059/1059), done.
Checking connectivity... done

mika@lusikka /tmp
  % cd omnisharp-emacs-test                                                                                                                                                                                  

mika@lusikka /tmp/omnisharp-emacs-test [master]
± % git checkout origin/melpa-testing                                                                                                                                                                        
Note: checking out 'origin/melpa-testing'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at ed1e675... testing

mika@lusikka /tmp/omnisharp-emacs-test [ed1e675]
± % ./run-melpa-build-test.sh                                                                                                                                                                                
Cloning into 'melpa'...
remote: Counting objects: 16526, done.
remote: Total 16526 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (16526/16526), 4.47 MiB | 1.38 MiB/s, done.
Resolving deltas: 100% (8374/8374), done.
Checking connectivity... done
 • Removing package sources ...
rm -rf ./working/*
 • Removing packages ...
rm -rfv ./packages/*
removed ‘./packages/index.html’
 • Removing json files ...
rm -vf html/archive.json html/recipes.json
 • Removing sandbox files ...
if [ -d './sandbox' ]; then \
        rm -rfv './sandbox/elpa'; \
        rmdir './sandbox'; \
    fi
 • Building recipe omnisharp ...
/usr/bin/timeout -k 60 600 emacs --no-site-file --batch -l package-build.el --eval "(let ((package-build-stable nil) (package-build-write-melpa-badge-images t) (package-build-archive-dir (expand-file-name \"./packages\" pb/this-dir))) (package-build-archive 'omnisharp))"

;;; omnisharp

Fetcher: github
Source: OmniSharp/omnisharp-emacs

Cloning git://github.com/OmniSharp/omnisharp-emacs.git to /tmp/omnisharp-emacs-test/melpa/working/omnisharp/
Note: this :files spec is equivalent to the default.
/tmp/omnisharp-emacs-test/melpa/working/omnisharp/example-config-for-evil-mode.el -> /tmp/omnisharp23276NmN/omnisharp-20141201.2121/example-config-for-evil-mode.el
/tmp/omnisharp-emacs-test/melpa/working/omnisharp/omnisharp.el -> /tmp/omnisharp23276NmN/omnisharp-20141201.2121/omnisharp.el
Wrote /tmp/omnisharp-emacs-test/melpa/packages/omnisharp-readme.txt
File: /tmp/omnisharp-emacs-test/melpa/packages/omnisharp-20141201.2121.entry
Contacting host: img.shields.io:80
img.shields.io/80 Name or service not known
make: [recipes/omnisharp] Error 255 (ignored)
 ✓ Wrote 4,0K -rw-r--r-- 1 mika mika 228 joulu  1 23:11 ./packages/omnisharp-20141201.2121.entry
 12K -rw-r--r-- 1 mika mika 10K joulu  1 23:11 ./packages/omnisharp-20141201.2121.tar
4,0K -rw-r--r-- 1 mika mika 414 joulu  1 23:11 ./packages/omnisharp-readme.txt 
 Sleeping for 0 ...
sleep 0

Contacting host: stable.melpa.org:80
Saving file /home/mika/.emacs.d/elpa/archives/melpa-stable/archive-contents...
Loading vc-git...
Wrote /home/mika/.emacs.d/elpa/archives/melpa-stable/archive-contents
Contacting host: melpa.org:80
Saving file /home/mika/.emacs.d/elpa/archives/melpa/archive-contents...
Wrote /home/mika/.emacs.d/elpa/archives/melpa/archive-contents
installing file /tmp/omnisharp-emacs-test/melpa/packages/omnisharp-20141201.2121.tar
Parsing tar file...
Parsing tar file...done
Extracting omnisharp-20141201.2121/example-config-for-evil-mode.el
Wrote /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/example-config-for-evil-mode.el
Extracting omnisharp-20141201.2121/omnisharp-pkg.el
Wrote /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp-pkg.el
Extracting omnisharp-20141201.2121/omnisharp.el
Wrote /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp.el
Making version-control local to omnisharp-autoloads.el while let-bound!
Generating autoloads for example-config-for-evil-mode.el...
Generating autoloads for example-config-for-evil-mode.el...done
Generating autoloads for omnisharp-pkg.el...
Generating autoloads for omnisharp-pkg.el...done
Generating autoloads for omnisharp.el...
Generating autoloads for omnisharp.el...done
Saving file /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp-autoloads.el...
Wrote /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp-autoloads.el
Checking /home/mika/.emacs.d/elpa/omnisharp-20141201.2121...
Compiling /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/example-config-for-evil-mode.el...

In toplevel form:
example-config-for-evil-mode.el:3:26:Warning: reference to free variable
    `omnisharp-mode-map'
example-config-for-evil-mode.el:38:7:Warning: assignment to free variable
    `omnisharp-auto-complete-want-documentation'

In end of data:
example-config-for-evil-mode.el:39:1:Warning: the following functions are not
    known to be defined: evil-define-key, omnisharp-unit-test
Wrote /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/example-config-for-evil-mode.elc
Compiling /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp-autoloads.el...
Compiling /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp-pkg.el...
Compiling /home/mika/.emacs.d/elpa/omnisharp-20141201.2121/omnisharp.el...

In toplevel form:
omnisharp.el:20:1:Error: Invalid function: --mapcat
Done (Total of 1 file compiled, 1 failed, 2 skipped)

Thanks in advance!

One of the things I like about CL is that it defines non-local exits in a lot of stuff (defuns, etc...)

I don't expect you like that... but if you do you might be interested in a patch to add blocks to the predicates and collection functions for the maps and reducers?

This would be useful as you could terminate the looping for a specific iteration if you wanted to, or perhaps for the whole thing if you wanted to.

What do you think?

See here about blocks and exits: http://www.gnu.org/software/emacs/manual/html_node/cl/Blocks-and-Exits.html#Blocks-and-Exits

This is often necessary but there's nothing in emacs for it:

(defun list-symbols (regex)
  (let ((lst))
    (mapatoms
     (lambda (a)
       (when (string-match-p regex (symbol-name a))
         (push a lst))))
    lst))

it's heavily related to dash core function, list processing, but it isn't exactly list processing per se. This is more of a very common generator function.

Is it worthwhile adding something like this?

Sorry I'm too lazy to do a PR. Basically, builtin sort is (sort list predicate). I'd like the dash version to be (-sort predicate list). Why? Because partial application, ->> etc. This order makes much much more sense than the built-in version.

(defun -sort (predicate list)
  "Same as `sort' with exchanged arguments."
  (sort list predicate))

Hi,

I noticed when releasing Ecukes 0.3 (depends on dash.el and s.el) that byte compiled files (which ELPA does automatically) does not support the --/it helpers. When I remove all *.elc and run again, it works.

An example reproduce this:

$ git clone git://github.com/rejeep/drag-stuff.git
$ cd drag-stuff
$ carton
$ carton exec elpa/ecukes-20121202.1208/ecukes features --dbg

cl has a pretty nice macro called destructuring-bind that lets you let-bind variables by pulling a data structure apart (a-la Clojure). Here's a typical example involving lists:

(destructuring-bind
      (first second)
    '(1 2)
  (format t "~%~%;;; => first:~a second:~a~&" first second))
;;; > first:1 second:2

I think that something like this will fit in dash.el nicely and will further reduce the need for mixing dash.el and cl.

The docstring of -mapcat talks about applying concat, so I assumed it would work with strings. However:

(-mapcat (lambda (x) (number-to-string x)) (list 1)) ;; "1"

but:

(-mapcat (lambda (x) (number-to-string x)) (list 1 2)) ;; (49. "2")

--mapcat actually uses append. I'd suggest that calling --mapcat --mapappend and making --mapcat use concat. However, that would break backwards compatibility, so I'm not sure what the best solution is here.

I tried flattening this list of message parts:

(#("multipart/mixed" 0 15
   (boundary "_002_29B227D89CBC7A47B50BF860CD7DB7BE0C50EE1FUKDC1PCMBX02wor_"
    buffer #<buffer  *mm*<80>>
    from "[email protected]" start nil))
  (#<buffer  *mm*<81>>
             ("text/plain" (charset . "us-ascii"))
             quoted-printable nil ("inline") nil nil nil)
  (#<buffer  *mm*<82>>
             ("application/msword" (name . "Deployment Engineer - v2.doc"))
             base64 nil
             ("attachment"
              (modification-date . "Fri, 08 Mar 2013 15:54:37 GMT")
              (creation-date . "Fri, 08 Feb 2013 10:09:55 GMT")
              (size . "34816")
              (filename . "Deployment Engineer - v2.doc"))
             "Deployment Engineer - v2.doc" nil nil))

It barfs on the cons cells:

  mapcar(#[(it) "�  !\207" [fn it] 2] (charset . "us-ascii"))
  -mapcat(-flatten (charset . "us-ascii"))
  -flatten((charset . "us-ascii"))
  #[(it) "� !\207" [fn it] 2]((charset . "us-ascii"))
  mapcar(#[(it) "�  !\207" [fn it] 2] ("text/plain" (charset . "us-ascii")))
  -mapcat(-flatten ("text/plain" (charset . "us-ascii")))
  -flatten(("text/plain" (charset . "us-ascii")))
  #[(it) "� !\207" [fn it] 2](("text/plain" (charset . "us-ascii")))
  mapcar(#[(it) "�  !\207" [fn it] 2] (#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil))
  -mapcat(-flatten (#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil))
  -flatten((#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil))
  #[(it) "� !\207" [fn it] 2]((#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil))
  mapcar(#[(it) "�  !\207" [fn it] 2] (#("multipart/mixed" 0 15 (boundary "_002_29B227D89CBC7A47B50BF860CD7DB7BE0C50EE1FUKDC1PCMBX02wor_" buffer #<buffer  *mm*<80>> from "[email protected]" start nil)) (#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil) (#<buffer  *mm*<82>> ("application/msword" (name . "Deployment Engineer - v2.doc")) base64 nil ("attachment" (modification-date . "Fri, 08 Mar 2013 15:54:37 GMT") (creation-date . "Fri, 08 Feb 2013 10:09:55 GMT") (size . "34816") (filename . "Deployment Engineer - v2.doc")) "Deployment Engineer - v2.doc" nil nil)))
  -mapcat(-flatten (#("multipart/mixed" 0 15 (boundary "_002_29B227D89CBC7A47B50BF860CD7DB7BE0C50EE1FUKDC1PCMBX02wor_" buffer #<buffer  *mm*<80>> from "[email protected]" start nil)) (#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil) (#<buffer  *mm*<82>> ("application/msword" (name . "Deployment Engineer - v2.doc")) base64 nil ("attachment" (modification-date . "Fri, 08 Mar 2013 15:54:37 GMT") (creation-date . "Fri, 08 Feb 2013 10:09:55 GMT") (size . "34816") (filename . "Deployment Engineer - v2.doc")) "Deployment Engineer - v2.doc" nil nil)))
  -flatten((#("multipart/mixed" 0 15 (boundary "_002_29B227D89CBC7A47B50BF860CD7DB7BE0C50EE1FUKDC1PCMBX02wor_" buffer #<buffer  *mm*<80>> from "[email protected]" start nil)) (#<buffer  *mm*<81>> ("text/plain" (charset . "us-ascii")) quoted-printable nil ("inline") nil nil nil) (#<buffer  *mm*<82>> ("application/msword" (name . "Deployment Engineer - v2.doc")) base64 nil ("attachment" (modification-date . "Fri, 08 Mar 2013 15:54:37 GMT") (creation-date . "Fri, 08 Feb 2013 10:09:55 GMT") (size . "34816") (filename . "Deployment Engineer - v2.doc")) "Deployment Engineer - v2.doc" nil nil)))
  eval((-flatten nic-parts) nil)
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp nil nil)

I was just looking at the code from the other PR, and it ocured to me, why not make -let's [] syntax match what clojure has?

See https://clojuredocs.org/clojure.core/let

It seems nice :)

While I agree that font-locking dash functions may be a good idea, it's not a good idea to force it on everyone, especially if the only reason they have dash installed is because some other package depends on it.

Could you make the font-locking optional, so that people who do want it can enable it with e.g. add-hook?

Hi. Since you use (eval list) in --reduce, it's impossible to byte compile it! This is a bit troubling when package.el tries to install the package that is using dash.

So, here's the "fixed" version.

(defmacro --reduce (form list)
  "Anaphoric form of `-reduce'."
  (let ((lv (make-symbol "list-value")))
    `(let ((,lv ,list))
       (if ,lv
           (--reduce-from ,form (car ,lv) (cdr ,lv))
         (let (acc it) ,form)))))

It evaluates the argument only once and bind it to a new local variable. Now, the whole let/if form is returned, but if only evaluates one branch, so it behaves the same as before, albeit being a bit longer.

Another idea for a useful function (I've implemented this while working on my current project)

I called it -split-by. I think it fits with the -split-* that it splits the list into two. Might be a bit confusing together with -split-with, but...

In haskell they call this partition, but you've already used that:

partition :: (a -> Bool) -> [a] -> ([a], [a])
The partition function takes a predicate a list and returns the pair of lists of elements which do and do not satisfy the predicate, respectively; i.e., 
partition p xs == (filter p xs, filter (not . p) xs)

Here's the elisp code

(defmacro --split-by (form list)
  "Anaphoric form of `-split-by'."
  (let ((l (make-symbol "left"))
        (r (make-symbol "right")))
    `(let (,l ,r)
       (--each ,list (if ,form (!cons it ,l) (!cons it ,r)))
       (list (nreverse ,l) (nreverse ,r)))))

(defun -split-by (pred list)
  "Returns a list of ((--filter PRED LIST) (--remove PRED LIST))."
  (--split-by (funcall pred it) list))

(-split-by (lambda (num) (= 0 (% num 2))) '(1 2 3 4 5 6 7)) ;; => '((2 4 6) (1 3 5 7))
(-split-by (lambda (num) (> 3 num)) '(7 5 3 5 1 3 2 1)) ;; => '((1 2 1) (7 5 3 5 3))
(--split-by (< it 5) '(3 7 5 9 3 2 1 4 6)) ;; => '((3 3 2 1 4) (7 5 9 6)) -- useful for quicksort :)
(-split-by 'cdr '((1 2) (1) (1 2 3) (4))) ;; => '(((1 2) (1 2 3)) ((1) (4))) -- split into lists of 1 element and lists of 2+ elements

I don't know how familiar you are with haskell, but Data.List is a good inspiraton: http://www.haskell.org/ghc/docs/7.0.4/html/libraries/base/Data-List.html

However, I think you have almost everything already done.

From 24.4:

In -table:
dash.el:955:25:Warning: value returned from (car v) is unused

In dash--match-cons-skip-cdr:
dash.el:1129:15:Warning: reference to free variable `s'
dash.el:1494:1:Error: Symbol's function definition is void: -all\?
Wrote /tmp/makepkg/emacs-dash/src/dash.el-2.10.0/dash-functional.elc

The (car v) warning goes away in trunk emacs.

There's butlast built-in fuction, which uses subroutines implemented in C, so it should be quite a lot faster. Is there any reason this wasn't used? (other than maybe you didn't know about it?)

In Python i can slice with step:

>>> a = [1,2,3,4,5,6]
>>> a[::2]
[1, 3, 5]

In dash.el it's not obvious how can i accomplish the same. Do you think adding a slice with step mechanism is a good idea?

My english level may be at fault, here, but I usually find -keep instead of -select -- I wish one had a link to the other in its docstring to help my memory, like:
See also: `-select'

WDYT ?

Salve!

I have a new version of --each, this one I called --each-when (following the similarly operating function --map-when. What this does is execute the body only on items of the list where they match the predicate. If they don't match the predicate, nothing happens.

it is "side effect" analogy of --map-when (and so could be emulated with it), but I find it more convenient and clean (--each-* clearly says it is only there for side-effect). It can also be implemented using (--each (--filter ...)) but that would be less optimal solution with additional clutter.

Here's the code:

(defmacro --each-when (list pred &rest body)
  "Anaphoric form of `-each-when'."
  (let ((l (make-symbol "list")))
    `(let ((,l ,list))
       (while ,l
         (let ((it (car ,l)))
           (when ,pred ,@body))
         (!cdr ,l)))))

(defun -each-when (list pred fn)
  "Calls FN with every item in LIST for which PRED is non-nil.
Returns nil, used for side-effects only."
  (--each-when list (funcall pred it) (funcall fn it)))

Some examples:

(let (s) (--each-when '(1 2 3 4 5 6) (= 0 (% it 2)) (!cons it s)) s) ;; '(6 4 2)
(let ((l '((1 . "a") (2 . "b") (3 . "c")))) (--each-when l (> (car it) 1) (setcdr it "changed")) l)
;; ((1 . "a")
;;  (2 . "changed")
;;  (3 . "changed"))

These examples are a bit contrived, but I've found a real use for it in my project, so I guess it can be useful for other people :P

Cheers!

Would that make sense to flag all the public functions as autoload?

In Python i can zip arbitrary number of lists:

>>> zip(range(3), range(3,6), range(6,9))
[(0, 3, 6), (1, 4, 7), (2, 5, 8)]

How do i do that in dash.el? See also: #14

The anaphoric functions makes inline functions nice and concise, but (lambda ..) can feel a bit clunky to use elsewhere. What about a -fn (or -f) macro:

(-fn "abc")     ;; => (lambda (&rest %&) "abc")
(-fn (* % %))   ;; => (lambda (%1 &rest %&) (* %1 %1)) -- % is translated to %1
(-fn (* %1 %2)) ;; => (lambda (%1 %2 &rest %&) (* %1 %2))
(-fn (* %2 %3)) ;; => (lambda (%1 %2 %3 &rest %&) (* %2 %3))

For extra sugar, (-fn f x y) could expand to (-fn (f x y)) if (functionp f).

I'd find it very useful to have a slice function that returns a copy of the list between two indices. Would you consider this to be a worthwhile addition?

I've written a basic implementation: https://gist.github.com/Wilfred/5020622 but happy to send a pull request if you're willing to add it.

This is another useful abstraction and a fairly common pattern. Works like map but also recursively maps the function to sublists (that is, a "tree" represented by nested lists).

;; helper function
(defun -cons-pair? (con)
  "Return non-nil if CON is true cons pair.
That is (A . B) where B is not a list."
  (and (listp con)
       (not (listp (cdr con)))))

;; the argument order: apply FN to TREE then apply FOLDER, logical :P
(defun -tree-map (fn tree &optional folder)
  "Apply FN to each element of TREE, and make a list of the results.
If elements of TREE are lists themselves, apply FN recursively to
elements of these nested lists.

If optional argument FOLDER is non-nil apply this function to the
result of applying FN to each list.  This will convert cons pairs
to lists before applying the FOLDER."
  (cond 
   ((not tree) nil)
   ;; "real" cons pair (a . b), that is `b' is not a list.  This still
   ;; wouldn't work with e.g. (1 2 3 . 4).  ((1 2 3) . 4) is fine.
   ((-cons-pair? tree)
    (let ((res (cons (-tree-map fn (car tree) folder)
                     (-tree-map fn (cdr tree) folder))))
      (if folder (apply folder (car res) (cdr res) nil) res)))
   ((listp tree)
    (let ((res (mapcar (lambda (x) (-tree-map fn x folder)) tree))) 
      (if folder (apply folder res) res)))
   (t
    (funcall fn tree))))

(defmacro --tree-map (form tree &optional folder)
  "Anaphoric form of `-tree-map'."
  `(-tree-map (lambda (it) ,form) ,tree ,folder))

;; examples
(-tree-map '1+ '(1 (2 3) 4)) ;; => (2 (3 4) 5)
(--tree-map (+ 2 it) '(1 (2 3) 4)) ;; => (3 (4 5) 6)
(--tree-map (case it (+ '-) (- '+) (t it)) '(+ 1 2 (- 4 5) (+ 3 4))) ;; => (- 1 2 (+ 4 5) (- 3 4))

This can be used to implement other functions, such as:

;; this version can also flatten alists. SUPER HANDY!
(defun -flatten (list)
  "Take a nested list LIST and return its content as a single, flat list."
  (-tree-map 'list list 'append))

(-flatten '(("(" . ")") ("[" . "]") ("{" . "}") ("`" . "'"))) ;; => ("(" ")" "[" "]" "{" "}" "`" "'")

;; possible name: -clone
(defun -copy (list)
  "Create a deep copy of LIST.
The new list has the same elements but all cons are replaced with
new ones.  This is useful when you need to clone a structure such
as plist or alist."
  (-tree-map 'identity list))

(defmacro -R (&rest forms)
  "Return a function that applies FORMS to (list it)."
  `(lambda (&rest it) ,@forms))

(defun -tree-reverse (tree)
  "Reverse the TREE while preserving the structure."
  (-tree-map 'identity tree (-R (nreverse it))))

(-tree-reverse '((1 (2 3)) (4 5 6))) ; ;; => ((6 5 4) ((3 2) 1))

;; some more examples
(-tree-map 'identity '((1 4 5) (0 (1 2 (3 (3 4) 4)))) '+) ;; => 27
(--tree-map (list (+ 2 it)) '((1 4 5) (0 (1 2 (3 (3 4) 4)))) 'append) ;; => (3 6 7 2 3 4 5 5 6 6)
;; number of items in tree
(--tree-map 1 '((1 4 5) (0 (1 2 (3 (3 4) 4)))) '+) ;; => 10
;; tree depth
(--tree-map 0 '((1 4 5) (0 (1 2 (3 (3 4) 4)))) (-R (1+ (apply 'max it)))) ;; => 5

;; generate HTML from sexps!
(-tree-map 'identity '(html 
                       (head (title "Hello! " "this is" " my homepage"))
                       (body 
                        (p "Welcome, this is " (b "amazing") " fun")
                        (p "Second paragraph."))) 
           (-R (concat
                "<"
                (symbol-name (car it))
                ">"
                (apply 'concat (cdr it))
                "</"
                (symbol-name (car it))
                ">"
                )))

;; output split for readability
;; "<html><head><title>Hello! this is my homepage</title></head>
;; <body><p>Welcome, this is <b>amazing</b> fun</p><p>Second paragraph.</p></body></html>"

Especially the -copy/-clone function is really handy if you work a lot with plists. Because their internal crazy rules for updates... it's just safer to do a copy each time.

Hey Magnars,

Im looking for a function that adds an element to a list.

Something like:

(-insert 'x 1 '(a b c))
=> '(a x b c)

Is there anything in dash.el for this? would you accept a pull request for it?

Thanks,

Joel