[Feature]: Reduce size of repository after pulling changes about elpaca

[progfolio]

I don't want packages to grow in size over time.

Elpaca is designed to be extensible.
A custom build step which reduces the size of the repository can be added.
See +elpaca-prune-repo in the following test case for a rough idea:

Test Case

How to run this test?

(elpaca-test
  :early-init (setq elpaca-menu-functions nil)
  :init (defun +elpaca-prune-repo (e)
          "Prune E's repo."
          (elpaca--signal e "Pruning repo")
          (let ((default-directory (elpaca<-repo-dir e)))
            (elpaca-process-call "git" "pull" "--depth" "1")
            (elpaca-process-call "git" "gc" "--prune=now"))
          (elpaca--continue-build e))
  (add-to-list 'elpaca-build-steps #'+elpaca-prune-repo 'append)
  (elpaca (doct :depth nil :host github :repo "progfolio/doct"))
  (elpaca-wait)
  (elpaca-with-dir 'doct repo
    (print (string-trim (elpaca-process-output "git" "log")))))

Host Env

elpaca	3cadcad HEAD -> master, origin/master, origin/HEAD
installer	0.6
emacs	GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo version 1.18.0) of 2024-02-28
git	git version 2.44.0

Output

Elpaca menu item cache discarded due to version change.
  INFO     Scraping files for loaddefs... 
  INFO     Scraping files for loaddefs...done
  GEN      ../elpaca-autoloads.el
Cloning into '/tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca'...
Your branch is up to date with 'origin/master'.
Checking /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-info.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-log.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-manager.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-menu-elpa.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-menu-melpa.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-menu-org.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-process.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-test.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca-ui.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/elpaca.el...
Checking /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/doc...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/doc/early-init.el...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/doc/init.el...
Checking /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/extensions...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/extensions/elpaca-use-package.el...
Checking /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/images...
Checking /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/test...
Compiling /tmp/elpaca.fZ5jgZ/elpaca/repos/elpaca/test/elpaca-test.el...
Done (Total of 11 files compiled, 3 skipped in 4 directories)


"commit 5cab660dab653ad88c07b0493360252f6ed1d898
Author: Nicholas Vollmer <[email protected]>
Date:   Thu Jun 22 14:47:30 2023 -0400

    add support for \"here\" target
    
    Added in Org 9.7"


 Test Env

Elpaca 3cadcad grafted, HEAD -> master, origin/master, origin/HEAD
installer:      0.7
emacs-version:  GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo version 1.18.0)
 of 2024-02-28
git --version:  git version 2.44.0

Does that help?

[amano-kenji]

Gentoo linux keeps fetch depth at 1 by default for all package repositories. I think fetch depth of 1 should be the default, and any deviation should be configurable.

Elpaca should also account for forced git pushes.

I don't think people care about past commits in most cases. What casual users want out of the box should be the default.

I don't want to worry about limitless growth of files and directories. In the past, I was burned by log files that weren't rotated. They filled the entire partition and broke my system. They also take up space that I can use for other things.

[progfolio]

I think fetch depth of 1 should be the default, and any deviation should be configurable.
I don't want to worry about limitless growth of files and directories.

Thanks for sharing your opinions.
I've added an optional COMMAND parameter to elpaca--fetch so you could delegate to it with your preferred git command.
e.g.

(define-advice elpaca--fetch (:around (fn &rest args) "shallow")
  "Shallow fetch E's remote."
  (apply fn (append args '("git" "fetch" "--depth=1"))))

I'd like to make this more flexible (ideally it should involve defining a new build step to replace elpaca--fetch sans advice), but that will involve a more significant redesign of how a package's build-steps are determined under different circumstances. Still working on a design for that.

I don't think people care about past commits in most cases.

I don't agree. People often, and understandably so, want to see what is being updated prior to updating.
Fetching only the tip of the remote does not allow one to review intermediate changes.
It also makes it more difficult to roll back changes.

What casual users want out of the box should be the default.

I'm not interested in implementing Gentoo's model as a default.
For most the size of the repository cache is a negligible trade-off.
Those who wish to optimize that can customize Elpaca similar to how I've suggested in this thread.

Elpaca should also account for forced git pushes.

For this rare case, a user who does not care about the history of the package can M-x elpaca-delete the package and re-install it.

[amano-kenji]

Okay. Can you at least document how to keep the commits shallow over time?

I think shallow fetch is common enough that it should be documented. Ideally, I'd like to see it on README or manual.

What about allowing users to configure fetch depth? I may want to keep the last 50 commits sometimes. Some people may want only one commit. By keeping the last N commits, you can keep storage usage under control. N can be 1 or higher.

[progfolio]

Okay. Can you at least document how to keep the commits shallow over time?
I think shallow fetch is common enough that it should be documented. Ideally,
I'd like to see it on README or manual.

Feel free to add the advice in this thread to the wiki.

What about allowing users to configure fetch depth? I may want to keep the last
50 commits sometimes. Some people may want only one commit. By keeping the last
N commits, you can keep storage usage under control. N can be 1 or higher.

That could also be accomplished via the advice I provided.
I'm trying to reduce the number of recipe keywords, so it's unlikely I'll be adding one for this particular use case.

[amano-kenji]

I'm trying to reduce the number of recipe keywords

According to manual, a recipe keyword goes into menus or orders. Manual doesn't say anything about advice, and I don't know much about advice. I am an emacs lisp beginner.

I really don't understand how define-advice fits in elpaca.

I think this is a sign that documentation needs to be improved. If I were you, I would mention the possibility to customize git commands.... in the manual. I don't really know the details, but it seems that you can mention advice can be added to some elpaca functions to customize elpaca's behavior.

[progfolio]

If I were you, I would mention the possibility to customize git commands.... in the manual. I don't really know the details

The manual assumes a certain level of familiarity with elisp. There is a separate Elisp manual which ships with Emacs. For example, advice is explained here:

https://www.gnu.org/software/emacs/manual/html_node/elisp/Advising-Functions.html

In the future advising the function will not be necessary.
So for now, the approach is documented here.

[amano-kenji]

In the future advising the function will not be necessary.

How will it not be necessary? Can you tell me?

[progfolio]

In the future advising the function will not be necessary.

How will it not be necessary? Can you tell me?

See my comment here: #271 (comment) and
the paragraph that starts with "I'd like to make this more flexible...".
I've not yet implemented that idea yet, so I can't give you anything more concrete at this time.