Coder Social home page Coder Social logo

topgit's Introduction

TopGit - A different patch queue manager


DESCRIPTION
-----------

TopGit aims to make handling of large amounts of interdependent topic
branches easier. In fact, it is designed especially for the case where
you maintain a queue of third-party patches on top of another (perhaps
Git-controlled) project and want to easily organize, maintain and submit
them - TopGit achieves that by keeping a separate topic branch for each
patch and providing some tools to maintain the branches.


INSTALLATION
------------
See the file INSTALL.


RATIONALE
---------

Why not use something like StGIT or Guilt or 'rebase -i' for maintaining
your patch queue?  The advantage of these tools is their simplicity;
they work with patch _series_ and defer to the reflog facility for
version control of patches (reordering of patches is not
version-controlled at all).  But there are several disadvantages - for
one, these tools (especially StGIT) do not actually fit well with plain
Git at all: it is basically impossible to take advantage of the index
effectively when using StGIT.  But more importantly, these tools
horribly fail in the face of a distributed environment.

TopGit has been designed around three main tenets:

	(i) TopGit is as thin a layer on top of Git as possible.  You
still maintain your index and commit using Git; TopGit will only
automate a few indispensable tasks.

	(ii) TopGit is anxious about _keeping_ your history.  It will
never rewrite your history, and all metadata is also tracked by Git,
smoothly and non-obnoxiously.  It is good to have a _single_ point when
the history is cleaned up, and that is at the point of inclusion in the
upstream project; locally, you can see how your patch has evolved and
easily return to older versions.

	(iii) TopGit is specifically designed to work in a distributed
environment.  You can have several instances of TopGit-aware
repositories and smoothly keep them all up-to-date and transfer your
changes between them.

As mentioned above, the main intended use-case for TopGit is tracking
third-party patches, where each patch is effectively a single topic
branch.  In order to flexibly accommodate even complex scenarios when
you track many patches where many are independent but some depend on
others, TopGit ignores the ancient Quilt heritage of patch series and
instead allows the patches to freely form graphs (DAGs just like Git
history itself, only "one level higher").  For now, you have to manually
specify which patches the current one depends on, but TopGit might help
you with that in the future in a darcs-like fashion.

A glossary plug: The union (i.e. merge) of patch dependencies is called
a _base_ of the patch (topic branch).

Of course, TopGit is perhaps not the right tool for you:

	(i) TopGit is not complicated, but StGIT et al. are somewhat
simpler, conceptually.  If you just want to make a linear purely-local
patch queue, deferring to StGIT instead might make more sense.

	(ii) When using TopGit, your history can get a little hairy
over time, especially with all the merges rippling through. ;-)


SYNOPSIS
--------

	## Create and evolve a topic branch
	$ tg create t/gitweb/pathinfo-action
	tg: Automatically marking dependency on master
	tg: Creating t/gitweb/pathinfo-action base from master...
	$ ..hack..
	$ git commit
	$ ..fix a mistake..
	$ git commit

	## Create another topic branch on top of the former one
	$ tg create t/gitweb/nifty-links
	tg: Automatically marking dependency on t/gitweb/pathinfo-action
	tg: Creating t/gitweb/nifty-links base from t/gitweb/pathinfo-action...
	$ ..hack..
	$ git commit

	## Create another topic branch on top of master and submit
	## the resulting patch upstream
	$ tg create t/revlist/author-fixed master
	tg: Creating t/revlist/author-fixed base from master...
	$ ..hack..
	$ git commit
	$ tg patch -m
	tg: Sent t/revlist/author-fixed
	From: [email protected]
	To: [email protected]
	Cc: [email protected]
	Subject: [PATCH] Fix broken revlist --author when --fixed-string

	## Create another topic branch depending on two others non-trivially
	$ tg create t/whatever t/revlist/author-fixed t/gitweb/nifty-links
	tg: Creating t/whatever base from t/revlist/author-fixed...
	tg: Merging t/whatever base with t/gitweb/nifty-links...
	Merge failed!
	tg: Please commit merge resolution and call: tg create
	tg: It is also safe to abort this operation using `git reset --hard`
	tg: but please remember you are on the base branch now;
	tg: you will want to switch to a different branch.
	$ ..resolve..
	$ git commit
	$ tg create
	tg: Resuming t/whatever setup...
	$ ..hack..
	$ git commit

	## Update a single topic branch and propagate the changes to
	## a different one
	$ git checkout t/gitweb/nifty-links
	$ ..hack..
	$ git commit
	$ git checkout t/whatever
	$ tg info
	Topic Branch: t/whatever (1 commit)
	Subject: [PATCH] Whatever patch
	Base: 3f47ebc1
	Depends: t/revlist/author-fixed t/gitweb/nifty-links
	Needs update from:
		t/gitweb/nifty-links (1 commit)
	$ tg update
	tg: Updating base with t/gitweb/nifty-links changes...
	Merge failed!
	tg: Please commit merge resolution and call `tg update` again.
	tg: It is also safe to abort this operation using `git reset --hard`,
	tg: but please remember you are on the base branch now;
	tg: you will want to switch to a different branch.
	$ ..resolve..
	$ git commit
	$ tg update
	tg: Updating t/whatever against new base...
	Merge failed!
	tg: Please resolve the merge and commit. No need to do anything else.
	tg: You can abort this operation using `git reset --hard` now
	tg: and retry this merge later using `tg update`.
	$ ..resolve..
	$ git commit

	## Update a single topic branch and propagate the changes
	## further through the dependency chain
	$ git checkout t/gitweb/pathinfo-action
	$ ..hack..
	$ git commit
	$ git checkout t/whatever
	$ tg info
	Topic Branch: t/whatever (1/2 commits)
	Subject: [PATCH] Whatever patch
	Base: 0ab2c9b3
	Depends: t/revlist/author-fixed t/gitweb/nifty-links
	Needs update from:
		t/gitweb/pathinfo-action (<= t/gitweb/nifty-links) (1 commit)
	$ tg update
	tg: Recursing to t/gitweb/nifty-links...
	[t/gitweb/nifty-links] tg: Updating base with t/gitweb/pathinfo-action changes...
	Merge failed!
	[t/gitweb/nifty-links] tg: Please commit merge resolution and call `tg update` again.
	[t/gitweb/nifty-links] tg: It is also safe to abort this operation using `git reset --hard`,
	[t/gitweb/nifty-links] tg: but please remember you are on the base branch now;
	[t/gitweb/nifty-links] tg: you will want to switch to a different branch.
	[t/gitweb/nifty-links] tg: You are in a subshell. If you abort the merge,
	[t/gitweb/nifty-links] tg: use `exit` to abort the recursive update altogether.
	[t/gitweb/nifty-links] $ ..resolve..
	[t/gitweb/nifty-links] $ git commit
	[t/gitweb/nifty-links] $ tg update
	[t/gitweb/nifty-links] tg: Updating t/gitweb/nifty-links against new base...
	Merge failed!
	[t/gitweb/nifty-links] tg: Please resolve the merge and commit.
	[t/gitweb/nifty-links] tg: You can abort this operation using `git reset --hard`.
	[t/gitweb/nifty-links] tg: You are in a subshell. After you either commit or abort
	[t/gitweb/nifty-links] tg: your merge, use `exit` to proceed with the recursive update.
	[t/gitweb/nifty-links] $ ..resolve..
	[t/gitweb/nifty-links] $ git commit
	[t/gitweb/nifty-links] $ exit
	tg: Updating base with t/gitweb/nifty-links changes...
	tg: Updating t/whatever against new base...

	## Clone a TopGit-controlled repository
	$ git clone URL repo
	$ cd repo
	$ tg remote --populate origin
	...
	$ git fetch
	$ tg update

	## Add a TopGit remote to a repository and push to it
	$ git remote add foo URL
	$ tg remote foo
	$ tg push -r foo

	## Update from a non-default TopGit remote
	$ git fetch foo
	$ tg -r foo summary
	$ tg -r foo update


USAGE
-----

The 'tg' tool has several subcommands:

tg help
~~~~~~~
	Our sophisticated integrated help facility.  Mostly duplicates
	what is below, except for adding summary Usage lines.

	# to list commands:
	$ tg help
	# to get help for a particular command:
	$ tg help <command>

tg create
~~~~~~~~~
	Create a new TopGit-controlled topic branch of the given name
	(required argument) and switch to it.  If no dependencies are
	specified (by extra arguments passed after the first one), the
	current branch is assumed to be the only dependency.

	After `tg create`, you should insert the patch description into
	the '.topmsg' file, which will already contain some prefilled
	bits.  You can set the 'topgit.to', 'topgit.cc' and 'topgit.bcc'
	git configuration variables (see `man git-config`) in order to
	have `tg create` add these headers with the given default values
	to '.topmsg'.

	The main task of `tg create` is to set up the topic branch base
	from the dependencies.  This may fail due to merge conflicts.
	In that case, after you commit the conflict resolution, you
	should call `tg create` again (without any arguments); it will
	detect that you are on a topic branch base ref and resume the
	topic branch creation operation.

	In an alternative use case, if '-r BRANCH' is given instead of a
	dependency list, the topic branch is created based on the given
	remote branch.

tg delete
~~~~~~~~~
	Remove a TopGit-controlled topic branch of the given name
	(required argument). Normally, this command will remove only an
	empty branch (base == head) without dependents; use '-f' to
	remove a non-empty branch or a branch that is depended upon by
	another branch.

	The '-f' option is also useful to force removal of a branch's
	base, if you used `git branch -D B` to remove branch B, and then
	certain TopGit commands complain, because the base of branch B
	is still there.

	IMPORTANT: Currently, this command will _NOT_ remove the branch
	from the dependency list in other branches. You need to take
	care of this _manually_.  This is even more complicated in
	combination with '-f' - in that case, you need to manually
	unmerge the removed branch's changes from the branches depending
	on it.

	See also `tg annihilate`.

	TODO: '-a' to delete all empty branches, depfix, revert

tg annihilate
~~~~~~~~~~~~~
	Make a commit on the current TopGit-controlled topic branch
	that makes it equal to its base, including the presence or
	absence of .topmsg and .topdeps.  Annihilated branches are not
	displayed by `tg summary`, so they effectively get out of your
	way.  However, the branch still exists, and `tg push` will
	push it (except if given the '-a' option).  This way, you can
	communicate that the branch is no longer wanted.

	Normally, this command will remove only empty branch
	(base == head, except for changes to the .top* files); use
	'-f' to annihilate a non-empty branch.

tg depend
~~~~~~~~~
	Change the dependencies of a TopGit-controlled topic branch.
	This should have several subcommands, but only `add` is
	supported right now.

	The `add` subcommand takes an argument naming a topic branch to
	be added, adds it to '.topdeps', performs a commit and then
	updates your topic branch accordingly.  If you want to do other
	things related to the dependency addition, like adjusting
	'.topmsg', prepare them in the index before calling `tg depend
	add`.

	TODO: Subcommand for removing dependencies, obviously

tg files
~~~~~~~~
	List files changed by the current or specified topic branch.

	Options:
	  -i		list files based on index instead of branch
	  -w		list files based on working tree instead of branch

tg info
~~~~~~~
	Show summary information about the current or specified topic
	branch.

tg patch
~~~~~~~~
	Generate a patch from the current or specified topic branch.
	This means that the diff between the topic branch base and head
	(latest commit) is shown, appended to the description found in
	the '.topmsg' file.

	The patch is simply dumped to stdout.  In the future, `tg patch`
	will be able to automatically send the patches by mail or save
	them to files. (TODO)

	Options:
	  -i		base patch generation on index instead of branch
	  -w		base patch generation on working tree instead of branch

tg mail
~~~~~~~
	Send a patch from the current or specified topic branch as
	email(s).

	Takes the patch given on the command line and emails it out.
	Destination addresses such as To, Cc and Bcc are taken from the
	patch header.

	Since it actually boils down to `git send-email`, please refer
	to the documentation for that for details on how to setup email
	for git.  You can pass arbitrary options to this command through
	the '-s' parameter, but you must double-quote everything.  The
	'-r' parameter with a msgid can be used to generate in-reply-to
	and reference headers to an earlier mail.

	WARNING: be careful when using this command.  It easily sends
	out several mails.  You might want to run

		git config sendemail.confirm always

	to let `git send-email` ask for confirmation before sending any
	mail.

	Options:
	  -i		base patch generation on index instead of branch
	  -w		base patch generation on working tree instead of branch

	TODO: 'tg mail patchfile' to mail an already exported patch
	TODO: mailing patch series
	TODO: specifying additional options and addresses on command
	      line

tg remote
~~~~~~~~~
	Register the given remote as TopGit-controlled. This will create
	the namespace for the remote branch bases and teach `git fetch`
	to operate on them. However, from TopGit 0.8 onwards you need to
	use `tg push`, or `git push --mirror`, for pushing
	TopGit-controlled branches.

	`tg remote` takes an optional remote name argument, and an
	optional '--populate' switch.  Use '--populate' for your
	origin-style remotes: it will seed the local topic branch system
	based on the remote topic branches.  '--populate' will also make
	`tg remote` automatically fetch the remote, and `tg update` look
	at branches of this remote for updates by default.

tg summary
~~~~~~~~~~
	Show overview of all TopGit-tracked topic branches and their
	up-to-date status ('>' marks the current topic branch,
	'0' indicates that it introduces no changes of its own,
	'l'/'r' indicates respectively whether it is local-only or has a remote mate,
	'L'/'R' indicates respectively if it is ahead or out-of-date with respect to its remote mate,
	'D' indicates that it is out-of-date with respect to its dependencies,
	'!' indicates that it has missing dependencies [even if they are recursive ones], and
	'B' indicates that it is out-of-date with respect to its base).

	This can take a long time to accurately determine all the
	relevant information about each branch; you can pass '-t' to get
	just a terse list of topic branch names quickly.  Alternately,
	you can pass '--graphviz' to get a dot-suitable output to draw a
	dependency graph between the topic branches.

	You can also use the '--sort' option to sort the branches using
	a topological sort.  This is especially useful if each
	TopGit-tracked topic branch depends on a single parent branch,
	since it will then print the branches in the dependency order.
	In more complex scenarios, a text graph view would be much more
	useful, but that has not yet been implemented.

	The --deps option outputs dependency information between
	branches in a machine-readable format.  Feed this to `tsort` to
	get the output from --sort.

	Options:
	  -i		Use TopGit metadata from the index instead of the branch
	  -w		Use TopGit metadata from the working tree instead of the branch

	TODO: Speed up by an order of magnitude
	TODO: Text graph view

tg checkout
~~~~~~~~~~~
	Switch to a topic branch.  You can use 'git checkout <branch>'
	to get the same effect, but this command helps you navigate
	the dependency graph, or allows you to match the topic branch
	name using a regular expression, so it can be more convenient.

	There following subcommands are available:

	    'tg checkout push'    Check out a branch that directly
	    		       	  depends on your current branch.

	    'tg checkout pop'     Check out a branch that this branch
	    		       	  directly depends on.

	    'tg checkout goto <pattern>'  Check out a topic branch that
	    		          matches <pattern>.  <pattern> is used as
	    		          a sed pattern to filter all the
	    		          topic branches.

	    'tg checkout next'    An alias for 'push'.
	    'tg checkout child'   An alias for 'push'.
	    'tg checkout'      	  An alias for 'push'.

	    'tg checkout prev'    An alias for 'pop'.
	    'tg checkout parent'  An alias for 'pop'.
	    'tg checkout ..'      An alias for 'pop'.

	If any of the above commands can find more than one possible
	branch to switch to, you will be presented with the matches
	and ask to select one of them.

	The <pattern> of 'tg checkout goto' is optional.  If you don't
	supply it, all the available topic branches are listed and you
	can select one of them.

	Normally, the 'push' and 'pop' commands moves one step in
	the dependency graph of the topic branches.  The '-a' option
	causes them (and their aliases) to move as far as possible.
	That is, 'tg checkout push -a' moves to a topic branch that
	depends (directly or indirectly) on the current branch and
	that no other branch depends on.  'tg checkout pop -a'
	moves to a regular branch that the current topic branch
	depends on (directly or indirectly).  If there is more than
	one possibility, you will be prompted for your selection.

tg export
~~~~~~~~~
	Export a tidied-up history of the current topic branch and its
	dependencies, suitable for feeding upstream.  Each topic branch
	corresponds to a single commit or patch in the cleaned up
	history (corresponding basically exactly to `tg patch` output
	for the topic branch).

	The command has three possible outputs now - either a Git branch
	with the collapsed history, a Git branch with a linearized
	history, or a quilt series in new directory.

	In the case where you are producing collapsed history in a new
	branch, you can use this collapsed structure either for
	providing a pull source for upstream, or for further
	linearization e.g. for creation of a quilt series using git log:

		git log --pretty=email -p --topo-order origin..exported

	To better understand the function of `tg export`, consider this
	dependency structure:

	origin/master - t/foo/blue - t/foo/red - master
	             `- t/bar/good <,----------'
	             `- t/baz      ------------'

	(where each of the branches may have a hefty history). Then

	master$ tg export for-linus

	will create this commit structure on the branch 'for-linus':

	origin/master - t/foo/blue -. merge - t/foo/red -.. merge - master
	             `- t/bar/good <,-------------------'/
	             `- t/baz      ---------------------'

	In this mode, `tg export` works on the current topic branch, and
	can be called either without an option (in that case,
	'--collapse' is assumed), or with the '--collapse' option, and
	with one mandatory argument: the name of the branch where the
	exported result will be stored.

	When using the linearize mode:

	master$ tg export --linearize for-linus

	you get a linear history respecting the dependencies of your
	patches in a new branch 'for-linus'.  The result should be more
	or less the same as using quilt mode and then reimporting it
	into a Git branch.  (More or less because the topological order
	can usually be extended in more than one way into a total order,
	and the two methods may choose different ones.)  The result
	might be more appropriate for merging upstream, as it contains
	fewer merges.

	Note that you might get conflicts during linearization because
	the patches are reordered to get a linear history.

	When using the quilt mode,

	master$ tg export --quilt for-linus

	would create the following directory 'for-linus':

	for-linus/t/foo/blue.diff
	for-linus/t/foo/red.diff
	for-linus/t/bar/good.diff
	for-linus/t/baz.diff
	for-linus/series:
		t/foo/blue.diff -p1
		t/bar/good.diff -p1
		t/foo/red.diff -p1
		t/baz.diff -p1

	With '--quilt', you can also pass the '-b' parameter followed
	by a comma-separated explicit list of branches to export, or
	the '--all' parameter (which can be shortened to '-a') to
	export them all. These options are currently only supported
	with '--quilt'.

	In '--quilt' mode the patches are named like the originating
	topgit branch.  So usually they end up in subdirectories of the
	output directory.  With the '--flatten' option the names are
	mangled so that they end up directly in the output dir (slashes
	are substituted by underscores).  With the '--strip[=N]' option
	the first 'N' subdirectories (all if no 'N' is given) get
	stripped off.  Names are always '--strip'ped before being
	'--flatten'ed.  With the option '--numbered' (which implies
	'--flatten') the patch names get a number as prefix to allow
	getting the order without consulting the series file, which
	eases sending out the patches.

	TODO: Make stripping of non-essential headers configurable
	TODO: Make stripping of [PATCH] and other prefixes configurable
	TODO: --mbox option to export instead as an mbox file
	TODO: support --all option in other modes of operation
	TODO: For quilt exporting, export the linearized history created in a
	      temporary branch---this would allow producing conflict-less
	      series

tg import
~~~~~~~~~
	Import commits within the given revision range into TopGit,
	creating one topic branch per commit. The dependencies are set
	up to form a linear sequence starting on your current branch -
	or a branch specified by the '-d' parameter, if present.

	The branch names are auto-guessed from the commit messages and
	prefixed by 't/' by default; use '-p <prefix>' to specify an
	alternative prefix (even an empty one).

	Alternatively, you can use the '-s NAME' parameter to specify
	the name of the target branch; the command will then take one
	more argument describing a _single_ commit to import.

tg update
~~~~~~~~~
	Update the current, specified or all topic branches with respect
	to changes in the branches they depend on and remote branches.
	This is performed in two phases - first, changes within the
	dependencies are merged to the base, then the base is merged
	into the topic branch.  The output will guide you on what to do
	next in case of conflicts.

	When '-a' is specified, updates all topic branches matched by
	<pattern>s (see `git-for-each-ref(1)` for details), or all if
	no <pattern> is given.

	After the update, if a single topic branch was specified, it is
	left as the current one; if '-a' was specified, it returns to
	the branch which was current at the beginning.

	If your dependencies are not up-to-date, `tg update` will first
	recurse into them and update them.

	If a remote branch update brings in dependencies on branches
	that are not yet instantiated locally, you can either bring in
	all the new branches from the remote using `tg remote
	--populate`, or only pick out the missing ones using `tg create
	-r` (`tg summary` will point out branches with incomplete
	dependencies by showing an '!' next to them).

	TODO: tg update -a -c to autoremove (clean) up-to-date branches

tg push
~~~~~~~
	If '-a' or '--all' was specified, pushes all non-annihilated
	TopGit-controlled topic branches, to a remote repository.
	Otherwise, pushes the specified topic branches - or the
	current branch, if you don't specify which.  By default, the
	remote gets all the dependencies (both TopGit-controlled and
	non-TopGit-controlled) and bases pushed to it too.  If
	'--tgish-only' was specified, only TopGit-controlled
	dependencies will be pushed, and if '--no-deps' was specified,
	no dependencies at all will be pushed.

	The remote may be specified with the '-r' option. If no remote
	was specified, the configured default TopGit remote will be
	used.

tg base
~~~~~~~
	Prints the base commit of each of the named topic branches, or
	the current branch if no branches are named.  Prints an error
	message and exits with exit code 1 if the named branch is not
	a TopGit branch.

tg log
~~~~~~
	Prints the git log of the named topgit branch - or the current
	branch, if you don't specify a name.

	NOTE: if you have merged changes from a different repository, this
	command might not list all interesting commits.

tg prev
~~~~~~~
	Outputs the direct dependencies for the current or named branch.

	Options:
	  -i		show dependencies based on index instead of branch
	  -w		show dependencies based on working tree instead of branch

tg next
~~~~~~~
	Outputs all branches which directly depend on the current or
	named branch.

	Options:
	  -i		show dependencies based on index instead of branch
	  -w		show dependencies based on working tree instead of branch

TODO: tg rename

IMPLEMENTATION
--------------

TopGit stores all the topic branches in the regular 'refs/heads/'
namespace (so we recommend distinguishing them with the 't/' prefix).
Apart from that, TopGit also maintains a set of auxiliary refs in
'refs/top-*'.  Currently, only refs/top-bases/ is used, containing the
current _base_ of the given topic branch - this is basically a merge of
all the branches the topic branch depends on; it is updated during `tg
update` and then merged to the topic branch, and it is the base of a
patch generated from the topic branch by `tg patch`.

All the metadata is tracked within the source tree and history of the
topic branch itself, in .top* files; these files are kept isolated
within the topic branches during TopGit-controlled merges and are of
course omitted during `tg patch`.  The state of these files in base
commits is undefined; look at them only in the topic branches
themselves.  Currently, two files are defined:

	'.topmsg': Contains the description of the topic branch in a
mail-like format, plus the author information, whatever Cc headers you
choose or the post-three-dashes message.  When mailing out your patch,
basically only a few extra mail headers are inserted and then the patch
itself is appended.  Thus, as your patches evolve, you can record
nuances like whether the particular patch should have To-list /
Cc-maintainer or vice-versa and similar nuances, if your project is into
that.  'From' is prefilled from your current `GIT_AUTHOR_IDENT`; other
headers can be prefilled from various optional 'topgit.*' git config
options.

	'.topdeps': Contains the one-per-line list of branches this
branch depends on, pre-seeded by `tg create`. A (continuously updated)
merge of these branches will be the "base" of your topic branch.

IMPORTANT: DO NOT EDIT '.topdeps' MANUALLY!!! If you do so, you need to
know exactly what are you doing, since this file must stay in sync with
the Git history information, otherwise very bad things will happen.

TopGit also automagically installs a bunch of custom commit-related
hooks that will verify whether you are committing the '.top*' files in a
sane state. It will add the hooks to separate files within the 'hooks/'
subdirectory, and merely insert calls to them to the appropriate hooks
and make them executable (but will make sure the original hook's code is
not called if the hook was not executable beforehand).

Another automagically installed piece is a '.git/info/attributes'
specifier for an 'ours' merge strategy for the files '.topmsg' and
'.topdeps', and the (intuitive) 'ours' merge strategy definition in
'.git/config'.


REMOTE HANDLING
---------------

There are two remaining issues with accessing topic branches in remote
repositories:

	(i) Referring to remote topic branches from your local repository
	(ii) Developing some of the remote topic branches locally

There are two somewhat contradictory design considerations here:

	(a) Hacking on multiple independent TopGit remotes in a single
	repository
	(b) Having a self-contained topic system in local refs space

To us, (a) does not appear to be very convincing, while (b) is quite
desirable for 'git-log topic' etc. working, and increased conceptual
simplicity.

Thus, we choose to instantiate all the topic branches of given remote
locally; this is performed by `tg remote --populate`. `tg update` will
also check if a branch can be updated from its corresponding remote
branch.  The logic needs to be somewhat involved if we are to "do the
right thing".  First, we update the base, handling the remote branch as
if it was the first dependency; thus, conflict resolutions made in the
remote branch will be carried over to our local base automagically.
Then, the base is merged into the remote branch and the result is merged
to the local branch - again, to carry over remote conflict resolutions.
In the future, this order might be adjustable on a per-update basis, in
case local changes happen to be diverging more than the remote ones.

All commands by default refer to the remote that `tg remote --populate`
was called on the last time (stored in the 'topgit.remote' git
configuration variable). You can manually run any command with a
different base remote by passing '-r REMOTE' _before_ the subcommand
name.


REFERENCES
----------

The following references are useful to understand the development of
topgit and its subcommands.

* tg depend:
  http://lists-archives.org/git/688698-add-list-and-rm-sub-commands-to-tg-depend.html


THIRD-PARTY SOFTWARE
--------------------

The following software understands TopGit branches:

* http://magit.github.com/magit/[magit] - a git mode for emacs

IMPORTANT: Magit requires its topgit mode to be enabled first, as
described in its documentation, in the "Activating extensions"
subsection.  If this is not done, it will not push TopGit branches
correctly, so it's important to enable it even if you plan to mostly use
TopGit from the command line.

topgit's People

Contributors

arvidjaar avatar bertwesarg avatar cederlys avatar greenrd avatar ij1 avatar imz avatar jonas avatar kvaneesh avatar madduck avatar marcweber avatar metzen avatar navytux avatar ringlej avatar tschwinge avatar ukleinek avatar vvv avatar wentasah avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

topgit's Issues

`tg next` errors

[user@work topgit]$ tg next
fatal: Not a valid object name t/annihilate:.topdeps
fatal: Not a valid object name t/base:.topdeps
fatal: Not a valid object name t/base-usage:.topdeps
fatal: Not a valid object name t/depend-usage:.topdeps
fatal: Not a valid object name t/ignore-emacs-autosave:.topdeps
fatal: Not a valid object name t/no-export-topmsg-topdeps:.topdeps
fatal: Not a valid object name t/refactor-summary:.topdeps

Merge tg-managed branch

A command to merge a tg-managed branch would be nice.

The minimum solution would be very simple: Just do a normal git merge, but leave out .topdep and .topmsg.

In addition, an up-to-date check should be done.

`tg info` shows bogus "Needs update" with 0 commits

This only seems to occur with TopGit branches that depend on one or more other TopGit branches:

[user@work topgit]$ tg info t/summary-sort-order
Topic Branch: t/summary-sort-order (11/6 commits)
Subject: Specify the sort order of tg summary.
Base: 3242b63
Remote Mate: origin/t/summary-sort-order
Depends: t/summary-deps-as-function
Needs update from:
        t/summary-deps-as-function (0/0 commit)

Add man page

The version currently in Ubuntu does not have a man page.

Therefore, a man page should be added.

Add a noff driver for tg-export

I usually want to preserve the history of work I do in topic branches, and show them in the exported history as merge commits.

So, after doing tg export --noff foo, the history shown by git log --first-parent foo..origin/master (assuming origin/master is the bottommost dependency) would be identical to what --collapse does, but doing the log without --first-parent would show the rebased history of each topic branch, filtering to remove the .topmsg and .topdeps files.

In other words, I want the result of such an export to look like I had sequentially:

  1. branched a topic branch from the bottom dependency
  2. worked on the topic branch
  3. did a merge --no-ff to merge it into the export branch
  4. repeated 1-3 until all the topic branches were merged in

This lets me push up a nice, tidy progression of work, preserving the individual history from each of the topic branches (albeit cleaned up, so they aren't filled with topgit's merge commits).

remote mate: bad revision

greenrd@exherbo ~/git/topgit $ ~/bin/tg info
Topic Branch: t/import-move-head (3/2 commits)
Subject: import: Support moving HEAD onto new topic branch
Base: d279e29
Remote Mate: github/t/import-move-head
fatal: bad revision 'refs/remotes/github/top-bases/t/import-move-head'
Depends: master
Up-to-date.

tg files in a project with submodules output spurious errors

Vulcan:~/src/fpco $ tg files
error: unable to find 15843c6387e3b137bfe02e3f2367da4ef911078b
error: unable to find dbaef15dcc6cfd7063fb3f1124aa0fd85a6796cd
error: unable to find 1f485fec006ea6eb2f54130bf875fb2053afdd4c
error: unable to find d674fe4c75da77fd5a2a97770ed403944e589f6a
error: unable to find 94eb62de862bdaf2d3949a53a5d057d9841e6a55
error: unable to find 15843c6387e3b137bfe02e3f2367da4ef911078b
error: unable to find dbaef15dcc6cfd7063fb3f1124aa0fd85a6796cd
error: unable to find 1f485fec006ea6eb2f54130bf875fb2053afdd4c
error: unable to find d674fe4c75da77fd5a2a97770ed403944e589f6a
error: unable to find 94eb62de862bdaf2d3949a53a5d057d9841e6a55

[FR] ability to specify a merge strategy for dependencies

I have a workflow where I must "git merge -s ours topic1 topic2 ..." into master (before releasing a package in ALTLinux (in Russian) which includes topic1, topic2, etc. as patches in the package. (Then the commit is tagged as the release, so that the patches can always be generated if you have Git history up to the tagged commit.)

I'd like to automate this workflow with TopGit, so that tg update would know that these dependencies should be merged with -s ours.

tg create should be able to save this additional information about the desired merge strategy in .topdeps, and tg update should follow this specification.

non-POSIX usage

local is used, which works in bash and dash but apparently not in ksh, which is used in OpenBSD.

It's pretty silly not to have a way of declaring local variables (because POSIX hasn't standardised that feature) so I propose either moving to

#!/bin/bash

or rewriting TopGit in a better language. I haven't decided which option to take yet.

topgit upstream status

While this repository seems to be not active within the last 1.5 years or so, there is a fork (not marked as a fork, but still) by @mackyle in https://github.com/mackyle/topgit, being more active in that time frame.

As the Fedora package maintainer I wonder whether I should switch upstreams.

There are a bunch of issues open here, and even pull requests, and I cannot easily see to which extent bugs have been solved and fixes incorporated in @mackyle's repo.

@greenrd, @mackyle: Any comments?

`tg update`: Recursive abort doesn't fully abort

I had to re-abort it myself:

[t/summary-sort-order] [t/graphviz-indicate-current] tg: Ok, you aborted the merge. Now, you just need 
to                                                                                                    
[t/summary-sort-order] [t/graphviz-indicate-current] tg: switch back to some sane branch using `git che
ckout`.                                                                                               
[t/graphviz-indicate-current] tg: You are in a subshell. If you abort the merge,
[t/graphviz-indicate-current] tg: use `exit 1` to abort the recursive update altogether.
[t/graphviz-indicate-current] tg: Use `exit 2` to skip updating this branch and continue.
[t/graphviz-indicate-current] exit 1
exit
[t/graphviz-indicate-current] tg: Ok, you aborted the merge. Now, you just need to
[t/graphviz-indicate-current] tg: switch back to some sane branch using `git checkout`.
[user@work topgit]$ 

make fails for my git version

$ git version
git version 1.8.1.5 (Apple Git-44)

$ make  prefix=/usr/local/Cellar/topgit/0.9-29-g7f09 install
[SED] tg
./tg precheck
./tg: line 29: [: (Apple: integer expression expected
tg: fatal: git version >=  1.7.7.2 required
make: *** [precheck] Error 1
$

tg import -d blah HEAD doesn't work

tg import switches to the dependency branch first, so HEAD is incorrectly interpreted as the HEAD of the base branch, not as the HEAD when tg import was run.

TopGit needs a test suite

There's a small but non-zero chance that I could start work on a test suite for TopGit. It's not much fun enhancing the existing code without one, since I have to constantly worry about introducing invisible breakage. Therefore I've filed this issue mainly to seek advice / preferences from the maintainer(s) on the choice of language and framework for a potential test suite.

I have fairly extensive experience of test frameworks in shell, Ruby, Python, Perl, Java etc. My personal opinion is that shell is a fairly horrible language to write tests in, and the few existing shell test frameworks are significantly inferior to alternatives in Ruby and Python. So my recommendation would be something in Ruby or Python. I also find RSpec much richer than Python's unittest, but I'm not religious about it. For example, I wrote a fairly comprehensive test suite for obs-service-tar_scm in Python using unittest, which automatically sets up git repositories as fixtures for the git tests, and it works nicely.

If there is any kind of desire to eventually rewrite TopGit in Ruby or Python (which again I personally believe there should be, but I appreciate that might be contentious, and of course it would be no small undertaking), then it would make sense to write the test suite in the language which would be anticipated for the rewrite - clearly it would not make sense to have the test suite in (say) Ruby and the implementation in Python, or vice versa.

Opinions welcome!

.topmsg should be optional

It's often too heavyweight to expect the user to write a .topmsg file; the combination of the topic branch name and the log messages for the "real" commits within that topic branch are often sufficient, especially when the topic branch only contains a single "real" commit.

That said, it's worthwhile keeping the functionality as long as it's optional - having a message which summarises the topic branch can be very useful when sharing more complex topic branches, and also for auto-generating patch series cover letters and github pull requests.

See also #38, in which I propose ditching the .topmsg file altogether.

Getting the current branch name fails in macOS

Hello,

I am using macOS 10.11.6, and I had trouble to use TopGit. The most basic tg info ended up into a not a TopGit-controlled branch when the branch was created via tg create.

After taking a pick in the source code, I end up finding the cause. The way the current branch name is inferred does not work with macOS'sed:

- ansible-consul git:(t/readme)
๐Ÿต  $ git symbolic-ref HEAD 2>/dev/null | sed 's#^refs/\(heads\|top-bases\)/##'
refs/heads/t/readme

I tried to fix the regex but without success, so I ended up installing gnu-sed via homebrew:

$ brew install gnu-sed --with-default-names

And now everything is working.

I believe it would be worth to make the regex also work on macOS, or making it explicit somewhere than GNU-sed is needed.

Now I am getting back to trying and playing with TopGit :) Thank you for your works!

needs rebase support

As mentioned in #38, topgit's third stated design tenet is a worthy goal:

(iii) TopGit is specifically designed to work in a distributed environment. You can have several instances of TopGit-aware repositories and smoothly keep them all up-to-date and transfer your changes between them.

However this focus seems to have resulted in a lack of focus for the opposite case, i.e. where your topic branches are mainly private. In this case, when a topic A changes, it makes more sense to rewrite the history of the other topics depending on that topic A, i.e. to update them via rebase rather than merge, since merge muddies the commit history unnecessarily.

In fact rebase sometimes even makes more sense in the use case of distributed, public topic branches too - as long as other consumers of those topics are suitably warned in advance of the expectation that history will be rewritten.

[design] .topmsg / .topdeps pollute commit history

.topmsg and .topdeps represent SCM meta-data, and so I believe it's a fundamental design mistake to include them in the commit history:

  • It violates the separation of concerns between storage of content (files) and metadata.
  • This pollution leads to extra noise appearing in the repo contents, and in every-day git commands and UIs.
  • Converting normal branches into topic branches requires rebasing the whole topic branch. (Worse, there isn't even a helper to do this in the general case - tg import creates a topic branch per commit which is usually undesirable.)
  • It prevents creation of a github pull request directly from a t/foo topic branch.
  • Changing .topmsg or .topdeps either creates more noise, or requires rebasing the whole topic branch.
  • Creating a topic branch requires two sets of edits: one for .topmsg, and one for the initial commit message.

Having said that, I can hazard a guess as to why this design decision was made: in order to adhere to the third stated tenet:

(iii) TopGit is specifically designed to work in a distributed environment. You can have several instances of TopGit-aware repositories and smoothly keep them all up-to-date and transfer your changes between them.

This is clearly a worthy goal, but I think the current implementation is the wrong way of going about it. (However the original author's comments on this decision are encouraging.) In fact topgit already stores other meta-data such as refs/top-bases/* outside the commit history. There are alternate approaches worth considering, e.g.

None of these will get fully and automatically transferred across remotes during using of normal git push / pull / fetch, but it wouldn't take much effort to fill in the gaps.

tg update sometimes doesn't stop if git fails

e.g. due to an editor/IDE holding the index lock, we may see:


fatal: Unable to create '/home/robin/git/backend/.git/index.lock': File exists.

If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.

but then tg update continuing.

In this case it was tg update -a, but I'm not sure that's relevant.

TopGit doesn't work in a linked git worktree

/usr/bin/tg: line 118: /home/robin/git/foo/.git/worktrees/foo.2/hooks/pre-commit+: No such file or directory
chmod: cannot access '/home/robin/git/foo/.git/worktrees/foo.2/hooks/pre-commit+': No such file or directory

No help for help

$ tg help help
tg: no help for help
TopGit v0.9 - A different patch queue manager
Usage: tg ( help [<command>] | [-r <remote>] (base|create|delete|depend|export|files|import|info|log|mail|next|patch|prev|push|remote|summary|update) ...)

`tg checkout pop` doesn't work after annihilating a branch

[user@work topgit]$ tg annihilate -f
[t/checkout c21b63d] TopGit branch t/checkout annihilated.
 7 files changed, 215 deletions(-)
 delete mode 100644 .topdeps
 delete mode 100644 .topmsg
 delete mode 100644 leaves.awk
 delete mode 100644 tg-checkout.sh
tg: If you have shared your work, you might want to run `tg push` now.
tg: Then you probably want to switch to another branch.
tg: You are still on t/checkout
[user@work topgit]$ tg push
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 499 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To [email protected]:greenrd/topgit.git
   197e8e5..c21b63d  t/checkout -> t/checkout
   18ad9b4..93191f9  refs/top-bases/t/checkout -> refs/top-bases/t/checkout
[user@work topgit]$ tg info
Topic Branch: t/checkout (15/6 commits)
fatal: Not a valid object name t/checkout:.topmsg
Base: 93191f9
Remote Mate: origin/t/checkout
fatal: Not a valid object name t/checkout:.topdeps
Up-to-date.
[user@work topgit]$ tg checkout pop
fatal: Not a valid object name t/checkout:.topdeps

Rename branches

topgit should support renaming branches (and update all local dependencies accordingly).

0.8: Please add options -h, --help and -V, --version

With 0.8:

$ tg -h
fatal: Not a git repository (or any parent up to mount point /srv/deb)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

$ tg --version
fatal: Not a git repository (or any parent up to mount point /srv/deb)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

Please add options -h,--help to display brief synopsis and program description. Consider also adding options -V,--version (GNU standard; option -v is best reserved for --verbose) to display program version number, license and author in easy to parse format, Like

VERSION LICENSE-ABBREV AUTHOR

0.8 GPL-2+ First Last <[email protected]>

The official License abbreviations, like GPL-2+, are defined at https://spdx.org/licenses/

`tg info` on an annihilated branch gives errors

[user@work topgit]$ tg annihilate -f
[t/base-usage 7f8d339] TopGit branch t/base-usage annihilated.
 4 files changed, 12 insertions(+), 44 deletions(-)
 delete mode 100644 .topdeps
 delete mode 100644 .topmsg
 rewrite tg-base.sh (73%)
tg: If you have shared your work, you might want to run `tg push` now.
tg: Then you probably want to switch to another branch.
tg: You are still on t/base-usage
[user@work topgit]$ tg info
Topic Branch: t/base-usage (9/6 commits)
fatal: Not a valid object name t/base-usage:.topmsg
Base: ad57379
fatal: Not a valid object name t/base-usage:.topdeps
Up-to-date.

lots of stale branches?

I just noticed that there seem to be quite a lot of stale branches representing unmerged topics, e.g. t/ready-for-integration so I'm just wondering what is the current status of topgit maintainership? This is certainly not intended as any kind of criticism, so please don't take it as such! But before getting too far into development I just wanted to know whether my pull requests have any chance of being merged ;-) I'm also worried about developing on top of a master which is substantially behind more recent development, since that could result in extra merge conflicts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.