simoncropp / githubsync

A tool to help synchronizing specific files and folders across GitHub hosted repositories

GitHubSync

A tool to help synchronizing specific files and folders across repositories

See Milestones for release notes.

.net API

NuGet package

https://nuget.org/packages/GitHubSync/

Usage

// Create a new RepoSync
var repoSync = new RepoSync(
    log: Console.WriteLine);

// Add source repo(s)
repoSync.AddSourceRepository(new(
    // Valid credentials for the source repo and all target repos
    credentials: octokitCredentials,
    owner: "UserOrOrg",
    repository: "TheSingleSourceRepository",
    branch: "master"));

// Add sources(s), only allowed when SyncMode == ExcludeAllByDefault
repoSync.AddBlob("sourceFile.txt");
repoSync.AddBlob("code.cs");

// Remove sources(s), only allowed when SyncMode == IncludeAllByDefault
repoSync.AddBlob("sourceFile.txt");
repoSync.AddBlob("code.cs");

// Add target repo(s)
repoSync.AddTargetRepository(new(
    credentials: octokitCredentials,
    owner: "UserOrOrg",
    repository: "TargetRepo1",
    branch: "master"));

repoSync.AddTargetRepository(new(
    credentials: octokitCredentials,
    owner: "UserOrOrg",
    repository: "TargetRepo2",
    branch: "master"));

// Run the sync
await repoSync.Sync(syncOutput: SyncOutput.MergePullRequest);

^{snippet source | anchor}

dotnet Tool

This tool allows reading the configuration from a file. This allows customization of the templates and repositories without having to recompile any code.

Installation

Ensure dotnet CLI is installed.

Install GitHubSync.Tool

dotnet tool install -g GitHubSync.Tool

Usage

Run against the current directory will use githubsync.yaml in the current directory:

githubsync

Run against a specific config file:

githubsync C:\Code\Project\sync.yaml

Configuration definition

The configuration format is yaml. There should be 1 to n number of templates and 1 to n number of (target) repositories.

templates:
  - name: [template name]
    url: [repository url of the template]
    branch: [branch to use, defaults to `master`]
    
repositories:
  - name: [repository name]
    url: [repository url of the target repository]
    branch: [target branch, defaults to `master`]
    autoMerge: [true / false, true is only used when user is allowed to merge PRs on the target repository]
    templates:
      - [list of template names to use in the order to apply]

Example

templates:
  - name: geertvanhorrik
    url: https://github.com/geertvanhorrik/repositorytemplate
    branch: master
  - name: catel
    url: https://github.com/Catel/RepositoryTemplate.Components
    branch: master
  - name: wildgums-components-public
    url: https://github.com/wildgums/RepositoryTemplate.Components.Public
    branch: master
repositories:
  - name: CsvHelper
    url: https://github.com/JoshClose/CsvHelper
    branch: master
    autoMerge: false
    templates:
      - geertvanhorrik
  - name: Catel
    url: https://github.com/catel/catel
    branch: develop
    autoMerge: true
    templates:
      - geertvanhorrik
      - catel
  - name: Orc.Controls
    url: https://github.com/wildgums/orc.controls
    branch: develop
    autoMerge: true
    templates:
      - geertvanhorrik
      - wildgums-components-public

This example will result in the following:

• CsvHelper => use geertvanhorrik
• Catel => use geertvanhorrik + catel (combined, so catel can override files)
• Orc.Controls => use geertvanhorrik + wildgums-components-public (combined, so wildgums-components-public can override files)

Icon

Sync designed by Edward Boatman from The Noun Project.

githubsync's Issues

Make Sync() optionally merge the created pull requests

Add SyncOutput.CreateAndMergePullRequest (Leveraging the Status API (octokit/octokit.net#104) to determine if it's safe to auto-merge)

Allow yaml configuration files to drive the sync engine

I've written a simple wrapper around this library that creates a configuration similar to the one below:

templates:
  - name: geertvanhorrik
    url: https://github.com/geertvanhorrik/repositorytemplate
    branch: master
  - name: catel
    url: https://github.com/Catel/RepositoryTemplate.Components
    branch: master
  - name: wildgums-components-public
    url: https://github.com/wildgums/RepositoryTemplate.Components.Public
    branch: master

repositories:
  - name: CsvHelper
    url: https://github.com/JoshClose/CsvHelper
    branch: master
    autoMerge: false
    templates:
      - geertvanhorrik

  - name: Catel
    url: https://github.com/catel/catel
    branch: develop
    autoMerge: true
    templates:
      - geertvanhorrik
      - catel

  - name: Orc.Controls
    url: https://github.com/wildgums/orc.controls
    branch: develop
    autoMerge: true
    templates:
      - geertvanhorrik
      - wildgums-components-public

This way, you can create large lists of synchronizations without having to modify any code. Would you be interested in having this in GitHubSync?

Fix source branch being ignored

PR+Merge branch doesnt get automatically deleted

When using the CLI tool to sync repos using autoMerge: true, it creates a branch with a PR for the changes, then automatically merges it (this works fine). However once it's done, it leaves the branch still in the repository.

After some time this builds up and it's a pain to manually delete them one by one (x15 repos)

can you add an option for keepMergeBranch: true/false ? or otherwise make it auto delete the branch once it's done? There's no reason for it to stay.

Thanks!

can i take ownership of this?

ping @andreasohlund

Create a mmbot command to sync

pbot sync {repo} //default branch

pbot sync {repo} target branch {branch} //targets the give branch

Ignore/Exclude specific files/glob's from sync

Hi

It would be really useful to me if i could exclude certain files from syncronising. For example we have many microservice API repo's, all of which use a Template Repository to sync common properties/github workflows. for the github workflows specifically is where we have an issue.

Because our template repository has a workflow to sync to all other repo's when changed, this workflow also then gets pushed to all repositories. meaning we need an ugly if repository == 'githubuser/repo' in each action.

What i propose is adding a list property to the configuration like so:

templates:
  - name: [template name]
    url: [repository url of the template]
    branch: [branch to use, defaults to `master`]
    exclude:
        - '.github/**/*'
        - 'my_super_secret_file.yml'
    
repositories:
  - name: [repository name]
    url: [repository url of the target repository]
    branch: [target branch, defaults to `master`]
    autoMerge: [true / false, true is only used when user is allowed to merge PRs on the target repository]
    templates:
      - [list of template names to use in the order to apply]

I did fork the project and try to add this myself and submit a PR, but i found the code structure very difficult to follow, so perhaps its easier for you to add? It would be really useful though :)

DefaultTemplateRepo.cs link in README is broken

Allow sync to support multiple source repositories

At the moment, RepoSync only supports a single source repository. Would you be interested if I add support for multiple source repositories?

E.g.

AddSourceRepository(string organization, string repository, string branch);

Then I can loop them backwards:

for (int i = _sourceRepos.Count - 1; i >= 0; i--)
{
    // clone files (if not already cloned)
}

Go through all repos and add all relevant ones

Dogfood syncomatic

Cannot create PR on external repository

Today I tried to include an "external" (read: I am not an owner) to the list of repositories to sync (even "just" creating a PR). However, when it needs to create the blob, it fails with a NotFound.

I think in such a case, it needs to fork instead.

clean up syncomatic branches

for branches we decide not to merge it would be good to have a feature to clean them up.

Perhaps delete all Syncomatic-* branches older than X days??

Or delete all but the most current?

Figure out if master or develop should be used when syncing

Query GH to check for a develp branch. Use that one if exists. Use master if not

Should syncomatic be merged into pbot?

Make synchronisation case-insensitive

I've did a a sync and the result was that both nuget.config and NuGet.config files were int the repo

Sync labels from RepoStandards to other repos

Currently the NSB repo is the template:

https://github.com/Particular/NServiceBus/labels