Suggestion: Merge efforts with evil-special-modes about evil-integrations

[Ambrevar]

Let's proceed as follows if you agree:

• Create a new repo with a good name we've agreed on.

• Re-use my init functions.

• Re-use my rationale.

• Re-use the bindings I have and that you don't.

  • calendar
  • custom
  • debugger
  • diff-mode
  • eshell
  • image
  • info
  • man
  • outline
  • proced
  • woman

• Re-use your bindings.

  • ag
  • bookmarks
  • cider
  • compile
  • dired
  • edebug
  • elisp-refs
  • flycheck
  • ggtags
  • ibuffer
  • ivy
  • macrostep
  • occur
  • p4
  • pass
  • prodigy
  • slime
  • vlf
  • xref

• Merge our bindings in common.

  • ansi-term / term
  • help
  • package-menu
  • profiler

• Fix some parts:

  • Remove evil-org, evil-org-agenda is on its way upstream.
  • Work out some of the few questions left un-answered in the rationale, mostly about filtering / sorting and marking.
  • Turn my README to an org file?

• Normalize the code base: bindings should use the same convetion, remove the helpers / macros, wrap everything in -set-keys functions, comments, etc.

• When all set, release on MELPA!

[jojojames]

I've created a new repo at https://github.com/jojojames/evil-collection. I'll split up some of the topics that's come up in here as this thread is getting big.

The groundwork you've laid out seems good to me, we can go with that.

As for those extra packages you have, those sound great to include to.

Another minor topic, since it's just 2 people for now, lets try to maintain a single line of history (e.g. rebase instead of merge). :)

I'll close this thread and we can transition to the new repo.

[jojojames]

@auwsmit Sounds like a great idea.

@Ambrevar

I like what you've done with evil-special-modes.

I just had a couple of points I wanted to make sure we're agreeable on to see
if a merge would work well.

We can also use this as just a jumping off point to discussing some of
the patterns the package can eventually use.

0. I'm curious what you think about custom/extra functions in this package.

   https://github.com/jojojames/evil-integrations/blob/master/evil-ansi-term.el

   Is an example of one. I'd like to think we can take a "where appropriate,
   add custom functions" approach but that might be a little too oppinionated.

1. Most packages have SPC do something. Because of the ubiquity of SPC for leader,
   I'd prefer that to be unbound by default.

   Magit, Diff mode, Edebug are some I know have SPC bound.

   This is oppinionated but I'm not sure I want to go in my personal config and unbind
   SPC for every package.

2. C-h should not be rebound.
   Binding it will probably be a disservice to any other Emacs-er trying out evil+integrations.

3. M-. and M-, should work fine for gtd/pop.
   I don't remember what vim's standard gtd/pop definitions are but I ended
   up using the default Emacs binds. That said, I don't find the initial suggestions
   unappealing either.

4. Keep the name evil-integrations.
   I'm not a fan of evil-special-modes as a name. :)

   I had an idea in the future to extract some common/base functions I'm using in

   https://github.com/jojojames/hydra-integrations

   which evil-integrations can leverage later. I'm not sure what'll happen if
   we end up merging out efforts. Admittedly, this is probably not a big deal.

   The actual mechanics of merging the work is a mystery to me too.
   (e.g. Who owns the repo, PRs or both members will have commit access?)
   Keeping it simple is important.

5. C-h/C-l should probably not be used. We should just stick with $/^/0 for horizontal movement.
   Like #2 but I think the regular horizontal movement keys will work just fine.

6. u/U to undo marks is probably fine. It seems most modes that use marks don't
   usually have an editing component. I might be wrong here though.

7. Sorting seems to be relatively rare (when compared to all available modes).
   The 'o' key will probably be overloaded in this case.

8. To keep the goals of this package within reach, we restrict the changes to vanilla Emacs modes.

   We should probably aim for all modes we can get our hands on. :)

9. I've been using the evilify macro from spacemacs but I'd like to use it
   less and only for really simple cases. I think hand editing each key
   will provide the best end experience.

10. There's a few packages where it's not truly possible to bind all the available
    vim keys to it. Notmuch is a big one in my mind and it's something I'm planning
    on extracting from my config and adding to evil-integrations. I'm expecting
    a few packages to have more of a 'me first' -> 'evil second' approach to them.
    I think Dired and maybe IBuffer are other similar packages where a 'me first' approach
    is more suitable.

[Ambrevar]

Wow, that's a lot of points of contention! Let's get the discussion started then!

0. Extra functions can be used when they are necessary to implement standard Vim behaviour.
   Example: EMMS has a "paste below" function, but it cannot paste above. So if we bind "paste below" to p, it's confusing that P does not paste above.
   In that case I think new functions are commended.

   See https://github.com/Ambrevar/dotfiles/blob/master/.emacs.d/lisp/init-evil-emms.el.

   On the other hand, any feature that enhances the mode in a non-Vim-exclusive way should be reported upstream.
   It's important to restrict ourselves to bindings only:

   • Else it will go out of hand.
   • And it will result in bloated packages that purist tend not to like.

1. The leader key is very often bound to , as well. In the end the leader key takes precedence, so it does not matter if we bind SPC to scroll-up for users of SPC-leaders.
   On the other hand, many users will expect SPC to scroll up. So it better be there.

2. C-h should not be rebound, I agree.

3. I think we should keep to Vim's standard just like for the other bindings. We ought to be consistent.
   The users are free to rebind M-. if they want to. It can be done globally, so it's no big deal.

   See http://vim.wikia.com/wiki/Go_to_definition_using_g.

4. I do not like the name evil-special-modes either, so we can go for evil-integration. I'd suggest without s though, I think it sounds weird in plural.
   I don't think we should use Hydra, in fact I think we'd be better off not using any framework. See below.

   About merging: There are different ways.
   We can work together via the usual PRs. We can also create a new organization with new repos.
   We could split the tasks as follows: one of us leads the vanilla mode bindings, the other one the third-party mode bindings.

5. OK for C-h/C-l, but $/^/0 will not always be sufficient. A good example is Helm, for which I use M-h and M-l.
   Helm is a tough one.

6. I cannot find any mode that can both "undo actions" and mark items.
   So we could go for u/U to unmark. My only fear is that we could run into such a pathological mode, in which case we would be force to set inconsistent bindings.

7. Sorting is not so rare, but admittedly not so common. What's your point though?
   I'm suggesting we define a "standard" binding in the rationale, so whenever sorting is available, its binding is consistently bound to o.
   If not available, then o can be used for anything.

8. Yes, but I'd rather set up separate repositories.

   It's a matter of packaging. Nobody wants a mode that drags bindings along for hundreds of packages they won't use.
   So I think we are better off with one repo for the vanilla modes, and individual repositories for third-party packages (as is already the case with Magit and Org).
   The more we cover the better of course.
   Community support will be invaluable here.

9. Agreed, and that connects with point 4: I don't think we need any framework here.
   Hand editing has the invaluable advantage of being the most transparent and easy to understand, for everyone.

10. I do not use Notmuch (I'm on mu4e) but from some of the comments on Reddit, it seems it's going to be a big one indeed.
    Also Gnus seems to be a tough one, and that's even more of a concern considering it's a vanilla mode.

    What do you mean with "me fist"?

[jojojames]

0. I think we're in agreement then.

1. On the other hand, many users will expect SPC to scroll up. So it better be there.

I think what might be missing is for people that expect to bind SPC but will then lose the key that was already on SPC.
I think Spacemacs gets around this by binding what would've been on SPC to '.

I wonder if it makes sense to bind it to SPC as well as '.

2. C-h should not be rebound, I agree.

   Cool.

3. I think we should keep to Vim's standard just like for the other bindings. We ought to be consistent.
   See http://vim.wikia.com/wiki/Go_to_definition_using_g.

   Hmnn, I didn't see a 'pop back from definition' key there. I think use gd or whatever makes sense for vim is fine as long as we don't override M-. and M-,.
   (They're somewhat similar to C-h to me.)

   I think we're in agreement here anyways.

4. I do not like the name evil-special-modes either, so we can go for evil-integration. I'd suggest without s though, I think it sounds weird in plural.

   Sure sounds good.

   I don't think we should use Hydra, in fact I think we'd be better off not using any framework. See below.

   I agree there, though as I was doing both hydra-integrations and evil-integrations, I was thinking there'd be a point where they'd probably use some similar common functions.
   We can consider this a non-factor though.

   About merging: There are different ways.

   Whatever works, let me know your preference and we can probably go with that.

   We could split the tasks as follows: one of us leads the vanilla mode bindings, the other one the third-party mode bindings.

   I've been doing my bindings in a very adhoc/scratch my own itch manner. Free for all (pick whichever modes) sounds good. :)

5. OK for C-h/C-l, but $/^/0 will not always be sufficient. A good example is Helm, for which I use M-h and M-l.
   Helm is a tough one.

   M-h and M-l sounds good to me.

6. I cannot find any mode that can both "undo actions" and mark items.
   So we could go for u/U to unmark. My only fear is that we could run into such a pathological mode, in which case we would be force to set inconsistent bindings.

   Yeah, we kind already see that with Magit with the u binding though. Where it's not really 'undo' but unstage. I see 'u' as more of a 'un' more than undo anyway.
   [Un]mark/[Un]do/[Un]stage, etc

7. Sorting is not so rare, but admittedly not so common. What's your point though?
   I'm suggesting we define a "standard" binding in the rationale, so whenever sorting is available, its binding is consistently bound to o.
   If not available, then o can be used for anything.

   That sounds good.

8. Yes, but I'd rather set up separate repositories.
   It's a matter of packaging. Nobody wants a mode that drags bindings along for hundreds of packages they won't use.
   So I think we are better off with one repo for the vanilla modes, and individual repositories for third-party packages (as is already the case with Magit and Org).
   The more we cover the better of course.
   Community support will be invaluable here.

   This might be a key difference here. I find one of the problems with the evil packages is you have to find every single one you want to integrate with.
   If I was a user from vim, I just want one package to set up that sets up all the defaults for me instead of manually figuring out each individual one.

   I have no problems with libraries with a ton of bindings if it doesn't impact anything (like startup/etc).

   The way it's being done now (after a specific package is loaded) seems like the right thing to do.

   Doing it this way improves the end experience in my opinion. If we split out extra packages, the workflow is:

   Install 3rd party package -> Use it -> Find it doesn't work with Evil -> Find Evil-Integration package -> Install it

   vs

   Install 3rd party package -> Use it -> Has Evil bindings automatically

   In this case, I'm a big fan of the latter. You can see this reflected in this repo, a lot of 3rd party packages are accounted for and I want to add more like notmuch for example.

9. Agreed, and that connects with point 4: I don't think we need any framework here.
   Hand editing has the invaluable advantage of being the most transparent and easy to understand, for everyone.

   Cool.

10. What do you mean with "me first"?

    I just mean some modes have so many keybindings that they kind of overwhelm the vim ones.
    Notmuch is probably the biggest example, Dired probably the other one.

[Ambrevar]

(1) ' is a useful binding, I don't think we should do that.

(3) In normal mode, M-. is bound to evil-repeat-pop-next.
How to you define 'pop back from definition'?

(4) I don't have any strong preferences. Ideally this should be part of Evil's organization.
An independent organization could create confusion in the community ("What is this other Evil?") so maybe we just stick to a simple repo, hosted by either you or me.
Most importantly: are you committed to maintaining it? If so, go for it. If you lack time or dedication, I can do it.
Also see point 8.

(8) Valid point.
Let me suggest a compromise: 1 repo for the vanilla modes, one repo for the non-vanilla modes.
Let's not forget that there are at least 4 repos out there already (Ediff, Magit, Org, mu4e).
We must get decided on this one. Maybe asking for feedback on Reddit would help getting decided here.

(10) We cannot reach perfection and it's fine if some modes like Notmuch have bindings that won't satisfy everyone.

[auwsmit]

Just adding my 2 cents:

0. I think small evil-helper-functions are a good idea when they allow the mode to act in a more intuitively vim-like way, as has been exemplified.

1. SPC is redundant with l in regular vim, so SPC should be bound where it makes sense. However, since SPC is also a very common leader (I use it), there should be alternative equivalent bindings (e.g. SPC & C-u to scroll-up) for those who to use it as their leader. This should appease both types of users.

   If having redundant mappings in this way takes up too many available keys, maybe a boolean variable evil-integration-space-alternative could decide whether SPC or its alternative is bound, so only one is ever in effect. Easy to mention as an optional in the README this way.

2. Agree with the current consensus, Emacs help is too universal to ever overwrite

3. Agree with the current consensus, g- mappings fit here

4. Agree with the current consensus, evil-integration is a good name. The plural doesn't bother me, but since "integration" is more of an action, singular does sound better.

   For merging, an organization sounds ideal. It simplifies who is the owner, and the new evil-integration repo would contain the best aspects of both projects.

5. I think mapping C-l to any sort of refresh/redisplay makes sense, and would open up r to do something more mnemonically useful. Also I don't mind using M-h/M-l to move in modes where $/^/0 are taken up.

6. I'm pretty into mnemonics, so I don't have any issue with u/U as unmark. It makes sense even if there may be some atypical mode that has unmarking and undoing, since it would just an exception to the rule. The benefit outweighs the small potential downside.

7. Agree with the current consensus, o/O as sort [o]rder in non-editing modes that require it is fine

8. I agree with jojo that it's sensible/convenient for the user to have as many modes as possible, so long as it doesn't affect startup or performance, which should be doable with something as straightforward as these rebindings.

   To be less intrusive, these 3rd party rebindings could be disabled by default, and/or they could have a separate init command from the Emace-native ones.

9. Agree with the current consensus, KISS

10. It may be so that some modes just have so many bindings that there is no ideal way to get it to perfectly follow the guidelines, but we'll just have to try as best we can, and make note of modes that seem to be impossible or require herculean efforts.

[Ambrevar]

(7) If you look at my rationale, you'll see that in Emacs s and S are usually used for sorting.
I tend to think it's more common, but I'm not sure. How does Vim stand with respect to sorting?

A much more action is filtering. In my rationale I've set it to s, but as I just said, that could be used for sorting.
Which key could we use for filtering then? o is a poor mnemonic...

(8) Startup would have zero impact since everything binding set can be wrapped within with-eval-after-load for its respective mode.

If you look at my setup, every mode has a <mode>-set-keys function, while the master file has an init function which registers all the *-set-keys function in their respective with-eval-after-load.

As @auwsmit just suggested, we can have 2 of those global init functions: one for the vanilla modes, one for the 3rd party. This comes at no cost.

[jojojames]

(1) ' is a useful binding, I don't think we should do that.

I like the idea by @auwsmit here with a redundant command. What do you think?

(3) In normal mode, M-. is bound to evil-repeat-pop-next.
How to you define 'pop back from definition'?

Pop back is when you go back from a 'go to definition' command. M-, is the standard
key command to do it in Emacs. M-. being 'go to definition'.

(4) I don't have any strong preferences. Ideally this should be part of Evil's organization.
An independent organization could create confusion in the community ("What is this other Evil?") so maybe we just stick to a simple repo, hosted by either you or me.
Most importantly: are you committed to maintaining it? If so, go for it. If you lack time or dedication, I can do it.
Also see point 8.

Lets go with a single repo for now. I'm up for hosting it for now.

More importantly, I think both of us should have commit access to the repository to keep it simple. I've invited you as a collaborator with commit access.
We can get started when ready. :)

(8) Valid point.
Let me suggest a compromise: 1 repo for the vanilla modes, one repo for the non-vanilla modes.
Let's not forget that there are at least 4 repos out there already (Ediff, Magit, Org, mu4e).
We must get decided on this one. Maybe asking for feedback on Reddit would help getting decided here.

As @auwsmit just suggested, we can have 2 of those global init functions: one for the vanilla modes, one for the 3rd party. This comes at no cost.

With this suggestion to use 2 separate commands to enable the vanilla & !vanilla modes, do we still need two separate repos?
1 will probably work just fine?

(10) We cannot reach perfection and it's fine if some modes like Notmuch have bindings that won't satisfy everyone.

Agreed.

Last comment on evilify macro. I think, while I want to move away from it. It still has a lot of value.
I don't have a problem with someone using the macro to generate an initial set of evil bindings for that mode first and then rewriting and editing it for a more precise experience later.

"It's better than nothing." is my consideration towards it.

[jojojames]

@auwsmit Re: C-l, probably fine as you described it. I really only care about the C-h binding. :)

[auwsmit]

So, I just ran into an issue with the name "evil-integration", which is that evil-mode already comes with a file called evil-integration.el with this description:

;;; evil-integration.el --- Integrate Evil with other modules

;; Author: Vegard Øye <vegard_oye at hotmail.com>
;; Maintainer: Vegard Øye <vegard_oye at hotmail.com>

So, I guess it needs an even more different name (is evil-integration-plus allowed? I'm not creative..) or else his permission would be needed to reuse the same name.

[Ambrevar]

(1) Redundant command: yes but which one? If it's only for SPC, why not. In most cases, SPC scrolls-up (@auwsmit: C-u scrolls half a page down), like C-f. The main difference is that in documentation modes like Info mode, SPC goes to the next node when it reaches the botton, while C-f (or Emacs' C-v) does not.

(3) So M-, is xref-pop-marker-stack and it's available in Evil. M-. is xref-find-definitions which is available in insert state, but in normal state, it's evil-repeat-pop-next.
Either way, this is an Evil's problem, I don't think it belongs to us to solve this.
This is how gd is used in my current bindings:

	evil-calendar.el:65:    "gd" 'calendar-goto-date ; "gd" in evil-org-agenda, "gd" in Emacs.
	evil-info.el:53:    "gd" 'Info-goto-node
	evil-man.el:47:    "gd" 'Man-goto-section
	evil-profiler.el:57:    "gd" 'profiler-report-find-entry

In those cases, the xref commands don't apply, so there is no problem. '' or C-o will do what you want.

(5) While C-l makes sense for refresh/redisplay because of terminals, I don't know if refreshing/redisplaying should be bound such a short binding, it's not something we always want to do, is it?
For now we have gr. r is used for other things.
It's a good suggestion though, I will include it to the rationale.

(8) 2 functions: That suggestion was in case we would go for a unique repo. Now that I've slept over it, I have another argument in favor of a unique repo: I think the reason the Evil project has lacked bindings for so many years is because of the fragmented effort. If we keep it together in one grouped effort, I believe it will create a strong dynamic.

1 or 2 init functions? Hmmm... this comes at zero cost, really. So what about 3? One for vanilla modes, one for non-vanilla modes, one that calls the 2 former! That way everybody is happy!

(11) Title: indeed, evil-integration is taken already. evil-integration-plus would be confusing.
What about evil-binding-collection, or even short, evil-collection?

(12) @noctuid made some points on my repo: Ambrevar/evil-special-modes#4
I think the most important point is that we should provide for key translations so that users of non-QWERTY key maps can adapt easily (e.g. Dvorak, Colemak). I think I've seen a key translation system somewhere before, maybe in Evil.

(13) Setup: to get started, we need to lay down some ground work on how we setup the bindings.
This is how I did it:

;; main file
(defun evil-special-modes-init ()
  (interactive)
  (with-eval-after-load 'calendar   (require 'evil-calendar) (evil-calendar-set-keys))
	;;...
	)

;; mode-specific file
(defun evil-calendar-set-keys ()
  (evil-define-key 'motion calendar-mode-map
    "h" 'calendar-backward-day
		;;...
		))

So it's very similar to your evil-integrations.el except it is wrapped in a function call, as per Emacs packaging standard (only requiring a package should not modify Emacs behaviour).

What do you think?

(14) This is a nit, really, but what about the key binding syntax? I see you use vectors and strings of escape control sequences like "\C-c\C-c". I used to do that, but then some sequence cannot be represented easily, like <backspace>.
I'm now using the following convention: write them like C-h c displays them. It's better for clarity I think, and we want any user to be able to read our bindings. Motto: be as transparent as possible.

• Without modifier not special keys, use a simple string, e.g. "gd".

• Anything else in (kbd ...). Use the <bracket> notation for special keys.

(15) Evilify macros: Why not using the expansion of those macros? ;)
I think it's important to keep our bindings as readable as possible to anyone. Consider there might be newcommers from Vim who do not know Elisp.
Last but not least, the macro does the easy part, that is, the most trivial bindings. Those are usually a matter of copy-pasting and can be done within minutes. The hard part cannot be derived automatically.
So the "better than nothing" can also be done with explicit binding very quickly I think.

[Ambrevar]

Other packages I have in my dotfiles:

• debbugs
• elfeed
• emms
• helm
• image+
• pdf-tools
• transmission
• ztree

[Ambrevar]

OK, I leave you to prepare the repo to avoid any live-conflict. Let me know when you are done.

[jojojames]

@Ambrevar It's fine to work on it at the same time, I was just thinking we want to rebase any changes if someone has already pushed to remote in order to keep a nice line of history.

Any changes you push, I'll just rebase on top before pushing my changes.