Currently, it fails with "Cannot get changes as this is a bare worktree repository"

Currently, some things are a bit hit-and-miss:

• Remotes
• Remote tracking branches
• Handling of local branches if a remote branch with the same name exists
• Subdirectories
• Branches with slashes

Remotes

Overview:

• If there is only one remote, this is implicitly used as the "default"
• If there is a remote defined as default (via default_remote), this is the "default"
• If there is more than one remote and no default, well, there is no default remote

In case we have default remote, everything is easy: That's the remote we use to set up remote tracking branches and pull/push/fetch from.

If there are multiple remotes without a default, the remote will always have to be specified for push/pull/fetch and tracking branches. I guess this is a very rare setup, so it will be fine to emit warnings and point users to default_remote.

• Verify the above with e2e tests
• Document the behaviour

Remote tracking branches

The desired behaviour is:

• The remote tracking branch is always the same name as the local name, except when either:
  • There is an explicit override with --track
  • There is a default prefix in the configuration file

When we create a worktree, there are a few different cases to handle:

• A local branch exists => use it. If it is set to track a remote branch that does not match the naming, emit a warning, but leave the remote tracking branch alone.
• A local branch does not exist, but a remote branch does => use it and set tracking
• Neither => Create a new local branch without tracking, except when tracking is configured (track.default = true)

• Verify the above with e2e tests
• Document the behaviour

Worktrees & Subdirectories

The rule is: Branch name == Worktree name. As a branch name can contain a slash, we cannot really handle subdirectories, as there would be no way to tell a subdirectory and a branch prefix apart. So subdirectories need to be removed.

• Remove subdirectory handling from worktrees
• Document the behaviour
• Make sure that subdirectories are removed together with the subdirectory

Right now, something like grm wt add dir1/dir2 will fails horribly. This should work and might make organization easier (e.g. a reviews directory or a features directory). See also #13.

Would be awesome be be able to just get a list of repos from GitLab/GitHub/Gitea whatever.

Something like

*grm repos sync --from github --token xxx
*grm repos sync --from gitlab --url https://gitlab.example.com --user mymyuser --group mygroup --token xxx
*grm repos sync --from gitea --url https://gitea.example.com --group mygroup --group myothergroup --token xxx

Additionally, grm repos find could also generate a list, so it can be tracked in version control. Maybe together with a diffing function that shows new/decommissioned repositoried on the remote side.

Open questions:

• Auth? API token via env variable? Something like --token-command "pass show github | head -1" to easily integrate with password managers?
• Which services? For me, only GitHub, GitLab and Gitea are interesting. If properly abstracted in the code, adding new services should be easy.
• Filtering for remote repositories to pull. Should be any combination of: user (pulling all visible repos of the specified user), group (pulling all visible repos of the group, recursively).

API clients:

• GitHub: https://github.com/XAMPPRocky/octocrab
• GitLab: https://crates.io/crates/gitlab
• Gitea: https://crates.io/crates/gitea

Right now, the command just immediately returns successfully, which is quite confusing.

Instead, there should be a warning like: "You did not specify any filters, so no repositories will be matched. See {--help} for a list of available filters"

• code layout in /src
• e2e test suite & considerations (regarding DRY, quality)

Great work, I really like the project and thank you for your initiative.

In reference to #48 it would be great if grm repos find local would not stop as soon as it finds a repository but it would continue to walk the directory tree to find other possible git repos. Some projects have a pattern of adding some directory to .gitignore and then putting sub repositories over there without adding them as submodules.

Otherwise (i.e. the default, simple), pushing will be denied:

fatal: The upstream branch of your current branch does not match the name of your current branch.

See this stackoverflow thread for more info

See

This depends on libgit2/libgit2#6327

So I have some projects that simply hold git repositories inside other git repositories (usually in .gitignore folders). And I want to keep that directory structure. Is this possible without introducing submodules?

• Check what other tools are already called GRM
• Check impact on URLs (github)

Otherwise, one has to first change into any workdir and do a fetch there. A simple grm wt fetch would be more elegant. This would also play nicely with the rebase (see #18). Something like grm wt rebase --fetch would be awesome.

Maybe https://docs.rs/camino/latest/camino/ is a good option?

Theoretically, git works with non-UTF8 files as well. But I think it would be fine to just bail on these kind of repositories.

Right now, they just stay in the worktree root afterwards.

• Warn about this?
• Move into .untracked & warn about this?

That'd would be helpful to identify what path triggers an error before a panic happens, a debug println!("{path}") could be added here

I needed this because of these lines:

Another option would be to return an error and handle it correctly in this block

Right now, e.g. when using the GitLab provider, the remote will be called gitlab. It's less surprising to use origin instead.

If required, it can still be overriden with --remote-name.

Right now, we refuse to pull when there are local changes. Automatic stashing/unstashing could make this a bit easier.

For example the command grm repos find local apache --format yaml > apache.yml creates my file as expected, however I have a problem.

The root is defined as ~/whatever/is/the/path which is not suitable in my case. I want it instead to be ./whatever/is/the/path meaning it should be relative to my current directory. Why? Well because I have a full repository right now that manages my other repositories, has a build script and other complex work. That repository needs to download stuff relative to its location, NOT the home directory.

So my suggestion is either to make it by default relative and you have to type the absolute path if you want to (preferred) OR alternatively pass a flag e.g. --relative to the find command

Take a look at the tree structure for example:

Permissions Size User  Date Modified Name
drwxr-xr-x     - taher  7 Jan 14:16  .git/
drwxr-xr-x     - taher 23 Dec  2022  apache/
drwxr-xr-x     - taher 25 Dec  2022  fork/
drwxr-xr-x     - taher  7 Jan 13:52  infra/
drwxr-xr-x     - taher 18 Dec  2022  moqui/
drwxr-xr-x     - taher 30 Dec  2022  nested/
drwxr-xr-x     - taher  3 Jan 13:54  project/
drwxr-xr-x     - taher  1 Jan 18:22  public/
drwxr-xr-x     - taher  7 Jan 13:52  taher/
.rw-r--r--    77 taher  7 Jan 13:53  .gitignore
.rw-r--r--   339 taher  7 Jan 14:15  apache.yml
.rw-r--r--  1.2k taher  7 Jan 13:46  build.gradle
.rw-r--r--   486 taher  7 Jan 14:15  fork.yml
.rw-r--r--  2.7k taher  7 Jan 14:15  infra.yml
.rw-r--r--  4.7k taher  7 Jan 14:16  moqui.yml
.rw-r--r--   19k taher  7 Jan 13:54  nested.yml
.rw-r--r--  1.5k taher  7 Jan 14:16  project.yml
.rw-r--r--   494 taher  7 Jan 14:16  public.yml
.rw-r--r--   450 taher 26 Dec  2022  README.md
.rw-r--r--   970 taher  7 Jan 14:16  taher.yml

Right now, the configuration files can contain additional keys without problems, they are just ignored. This is good for backwards compatibility during development, but can lead to confusing situation (e.g. when mistyping a key).

Enabling #[serde(deny_unknown_fields)] for those structs would reject those configuration files.

• Enable the setting
• Check the error codes

The errors are very unhelpful:

grm repos sync config -c ~/grm.toml
[✘] Error parsing configuration file "grm.toml": data did not match any variant of untagged enum Config

• "Future & Ideas" is out of date
• "Crates" is out of date
• No reference to YAML
• No mention of worktrees
• Add mirrors

Different remotes could be used at the same time, with the repositories being combined.

What information to use to determine whether two projects are the same on both forges?

Could be:

• ❓ Full path: Would only work in certain circumstances.
• ❌ Repository name: Not possible, as it's not unique per forge (repos with the same name can be in different groups)

There could also be a mapping that allows namespaces to be mapped between forges. Something like --map=github:github_user=gitlab=gitlab_user. Or in config:

[[mapping]]
github = "github_user"
gitlab = "gitlab_user"

But: What should be given to --group or --user then? Better to have a mapping from a forge-independent handle to a name for each forge. E.g:

--map=myuser@github:github_user@gitlab:gitlab_user --user=myuser

Or in config:

[mapping.myuser]
github = "github_user"
gitlab = "gitlab_user"

Whenever GRM then encounters a (namespace, name) tuple from a forge, it checks whether namespace or any parent matches the mapping, and replaces it accordingly.

E.g.:

Imagine the mapping from above:

This mapping would also transfer to the rest of GRM like local path names for clones.

This would also be cool to put into a global config in $XDG_CONFIG_HOME

E.g. grm wt convert master would automatically create a worktree from that branch after conversion.

But let's first see whether convert holds up to it's promises.

Currently, all branches that do not have changes are cleaned up. It may be desirable to keep those branches around until they are merged into the default branch.

Currently, the following behaviour is quite weird:

$ grm wt add x/branch

This creates a branch called branch (unexpected) inside the directory x/branch (expected). It's also weird, because the following will fail:

$ grm wt add x/branch
$ grm wt delete x/branch
[✘] Branch "branch" is checked out in worktree, this does not look correct

The docs say:

The branch inside the worktree is always the same as the directory name of the worktree.

So the branch name should be x/branch as well. Note that the worktree cannot contain slashes, so this may be a bit ugly to set up.

E.g. if there is a remote branch origin/foo, but no local branch foo, using grm wt add foo will create a new branch instead of reusing foo.

https://github.com/console-rs/indicatif

Git CLI and other git tool work, but grm reports error:

$ grm wt fetch
[✘] Error fetching remotes: failed to start SSH session: Unable to exchange encryption keys

Searched the error message online and it seems to relate to libgit2 version.

This would enable features like a "default-tracking", with a prefix. An example config:

default_tracking_enabled = true
default_tracking_prefix = "myprefix"

This would mean that a branch branch would be set up to track origin/myprefix/branch by default.

This could also be used to automatically convert repos if the config worktree_setup: true.

• Check whether the branch is merged into the default branch.

Currently, the .git file inside the worktrees references the main worktree via an absolute path. Same for the worktree symlinks in .git-main-working-tree/worktrees/{worktree}/commondir.

This means that whenever you move the whole worktree, the whole thing breaks.

Using git worktree actually leads to the same behaviour, so grm at least does not behave worse than git here.

I'm not sure whether using relative paths is something that git supports and libgit2 exposes. But would be really cool to have if possible. I'd rather not hack this manually, so if it's not exposed by libgit2, the behaviour just has to be accepted.

To do

• Check whether there is any way to make git use relative paths
• Check whether the functionality is there in libgit2
• Implement it in grm if possible. Relative paths should be the default. This would break moving single worktrees, but that is something that would not be supported anyway.
• Make it explicit in the docs that moving worktrees is not supported. If really necessary, there could be a separate command to do this. But I'm 99% certain that this is not required.

It's easier to have a 1-to-1 mapping between a configuration file and the respective tree of repositories. For multiple independent trees, separate configuration files can be used.

Right now, it's just unwrap() littered everywhere.

Credit to @jalr for that idea.

Write it down in a CONTRIBUTING.md and/or in docs.

Contribution workflow:

• Open issue for "bigger" changes / ideas so we can discuss them
• Make just check work
  • Formatting: cargo fmt
  • Linting: cargo clippy
  • Tests: See below
• Usual github pull request workflow against develop.
• Add yourself to CONTRIBUTORS if wanted

Testing:

• Unit / integ tests in rust (very small test suite) => just test-unit test-integration
• e2e test suite in e2e_tests. Should catch all scenarios that grm can be used in. Might make sense to write a test here that fails when making changes, to prevent regressions in the future. => just test-e2e

Automatically creates local worktrees for all (remote?) branches

I don't see a use case for having locally namespaced branches. For remote tracking branches, you're free to chose any name, including slashes.

See #55

Target OS: Linux (nothing else is tested).

The best way to go with the statically linked binary, which should work everywhere.

In that case, let's assume the first persistent branch is the default.

push.default = upstream is only necessary when the remote branch name differs from the local branch name. Otherwise, simple is much more ergonomic.

upstream leads to weird behaviour if there are multiple remotes. The problem is that only one remote can be the one with the upstream branch. Pushing to others without an explicit remote branch leads to the following error:

--- [origin] ---
fatal: You are pushing to remote 'origin', which is not the upstream of
your current branch 'develop', without telling me what to push
to update which remote branch.

It might make sense to print a warning in that case (multiple remotes with branch naming mismatch), which should be rare enough.

Currently, push.default is just set during worktree init/clone/conversion. In the future, this would also need to be done during each sync, as the number of remotes may change and the setting may have to be changed accordingly.

Also, the setting is currently repo-specific. But actually, it would be worktree-specific. Some worktrees might match the remote branch (master/main/develop being common), while others may not (I like to use a prefix with feature branches for example). Only the latter should use push.default = upstream when using multiple remotes.

As it's deeply nested, TOML was not a good choice. There is a lot of duplication and the section headers get quite verbose.

YAML is the better choice in hindsight.

Note that a possible global config would still use TOML.

A TOML in the root of a worktree like this:

persistent_branches = [
    "master",
    "develop",
]

[track]
default = true
default_remote = "origin"
default_prefix = "myprefix"

persistent_branches will be used for:

• grm wt clean will never clean up these branches
• grm wt clean will clean branches that are merged into these persistent branches
• grm wt delete does not require --force if the branch is merged into these persistent branches

default_track will automatically add a tracking branch on grm wt add. By default, it will track origin/{branchname}. The remote can be configured with default_track_remote, and a prefix can be given with default_track_remote. Prefix is used like {remote}/{prefix}/{branchname}

When using grm repos sync with a --group that does not exist, the output is just this:

Error: 404 Group Not Found

It should at least include the group name.

Currently, it defaults to a builtin name (e.g. gitlab for the GitLab provider).

Something like --remote-name origin would be good.

This almost never makes sense, and the default branch is almost always clean, so it's annoyingly always deleted.

Hi! I'm trying to build this with nix to test it out for a couple of fun scenarios/ideas.

I'm hitting this when building, with what I think is latest nightly:

git-repo-manager-unstable> error[E0015]: cannot call non-const fn `std::option::Option::<&str>::unwrap_or` in constants
git-repo-manager-unstable>   --> src/provider/github.rs:13:39
git-repo-manager-unstable>    |
git-repo-manager-unstable> 13 |     option_env!("GITHUB_API_BASEURL").unwrap_or("https://api.github.com");
git-repo-manager-unstable>    |                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
git-repo-manager-unstable>    |
git-repo-manager-unstable>    = note: calls in constants are limited to constant functions, tuple structs and tuple variants
git-repo-manager-unstable> error[E0015]: cannot call non-const fn `std::option::Option::<&str>::unwrap_or` in constants
git-repo-manager-unstable>   --> src/provider/gitlab.rs:12:68
git-repo-manager-unstable>    |
git-repo-manager-unstable> 12 | const GITLAB_API_BASEURL: &str = option_env!("GITLAB_API_BASEURL").unwrap_or("https://gitlab.com");
git-repo-manager-unstable>    |                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
git-repo-manager-unstable>    |
git-repo-manager-unstable>    = note: calls in constants are limited to constant functions, tuple structs and tuple variants

and if I nix derivation show ${grm} | grep rust-nightly-minimal then I see amongst the output:

/nix/store/6a1h550mcqhm12m7x8cbs6bs6qmbm7qy-rust-nightly-minimal-2023-04-28.drv

Any advice? Or if you know the last nightly that worked I might be able to pin it for now, I don't think it's recorded in the repo.

I wonder if you see any merit it putting names into using a slightly simplified (I think) configuration. This uses quite a few arrays as is that contain names for things that are already atomic. For example, directory names and remote names already require uniqueness.

In my view, this would make a slightly more terse and readable configuration in any format. For example, this config.toml:

[[trees]]
root = "~/projects"

[[trees.repos]]
name = "github.com/theherk/terraform-aws-apigateway-route-builder"
worktree_setup = false

[[trees.repos.remotes]]
name = "origin"
url = "[email protected]:theherk/terraform-aws-apigateway-route-builder.git"
type = "ssh"

[[trees.repos]]
name = "github.com/theherk/helix"
worktree_setup = false

[[trees.repos.remotes]]
name = "origin"
url = "[email protected]:theherk/helix.git"
type = "ssh"

[[trees.repos.remotes]]
name = "upstream"
url = "https://github.com/helix-editor/helix.git"
type = "https"

would become:

root = "~/projects"

["github.com".theherk.terraform-aws-apigateway-route-builder]
worktree_setup = false

["github.com".theherk.terraform-aws-apigateway-route-builder.remotes.origin]
type = "ssh"
url = "[email protected]:theherk/terraform-aws-apigateway-route-builder.git"

["github.com".theherk.helix]
worktree_setup = false

["github.com".theherk.helix.remotes.origin]
type = "ssh"
url = "[email protected]:theherk/helix.git"

["github.com".theherk.helix.remotes.upstream]
type = "https"
url = "https://github.com/helix-editor/helix.git"

That would have the byproduct of forcing names not be duplicated in these cases, and it feels a bit more nice to me.

Of course, you have put together a wonderful tool. Thank you.

Also, I think this could be further simplified using some very sensible heuristics to get the same configuration from:

root = "~/projects"

["github.com".theherk.terraform-aws-apigateway-route-builder]
worktree_setup = false

["github.com".theherk.helix]
worktree_setup = false

["github.com".theherk.helix.remotes.upstream]
url = "https://github.com/helix-editor/helix.git"