clever / microplane Goto Github PK

Dependabot Preview will be shut down on August 3rd, 2021.

It appears that this repository has configured updates for dep.

The GitHub-native version of Dependabot does not support dep, which was deprecated as of 2020.
Guidance from the dep maintainers is to upgrade to go mod, which Dependabot supports.

In order to keep getting updates for your Go dependencies, you'll need to migrate to go mod and add a Dependabot configuration for it to this repository.

When trying to push to a list of repositories sometimes it will fail with exit code 128. I believe this error is caused by insufficient access permissions while trying to clone. I understand why this is thrown and it can be resolved by updating the permissions but it would be nice if the failures would include which repo is throwing the error so the entire list doesn't have to be checked for permissions.

The snippet below shows output where part of a list has failed with exit code 128 but I have no idea which one is the problem. With a list of 10 repos this isn't a huge problem but when pushing to 200 repos it takes a very long time to check the permissions in each one.

Repo names removed as an abundance of caution

/home/runner/work/REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:21 cloning: REMOVED
2021/08/02 20:25:22 multiple errors: exit status 128 | exit status 128 | exit status 128 | exit status 128
Error: Process completed with exit code 1.

➜  a git:(master) ✗ mp init "org:15five"
2022/03/26 20:43:46 GET https://api.github.com/search/code?page=5&q=org%3A15five: 403 You have exceeded a secondary rate limit. Please wait a few minutes before you try again. []

When iterating through the results the program should sleep to avoid hitting a rate limit.
Workaround: using a more specific search. Although if a very large organization tried microplane they could run into the issue even with a specific search.

This error appears to happen when there is a repo where no changes have been made. This could be intentional, and as pushing should be allowed, or at least allowed with a flag like --skip-empty-changes turned on.

At least in GitLab, when an error occurs, the mp status output replaces the usual status output with the error text. This removes information useful in debugging (like the URL of the merge request).

One solution here would be to put the error message (if any) on a separate line so as to keep the status line available.

example-project		pushed		(merge error) status was not 'success', instead was 'failed'

Could become:

example-project 	pushed		status:?  assignee:dnesting https://gitlab.example.com/foo/example-project/-/merge_requests/123
					(merge error) status was not 'success', instead was 'failed'

I tried assigning a an entire team as a reviewer when doing mp push but the PR had no reviewers. Is there a special syntax or something I need? Does microplane handle assigning a team as a reviewer? 🤔 I'm not sure if it's a bug or if I'm missing something.

mp push -a teamname -r eks_cicd-util
mp push -a 15five/teamname -b 'Testing multi-repo changes with https://github.com/Clever/microplane. You can ignore this.'

Hello 👋

Thank you for this awesome project! I'd love to contribute to this project, and one thing that has bugged me a little is that I have no way to choose which kind of merge I'd like.

I would rather squash and merge than add a merge commit. I'm not sure if it's possible, but I'd love to have a go at it if that is a feature you'd like to see in Microplane!

Thanks again!

I have created a repos.txt with the following:

org/ops/test
org/ops/other/test

However when running mp init --provider=gitlab -f repos.txt this results in an error:

2021/05/27 11:06:05 unable determine repo from line, expected format '{org}/{repo}': org/ops/test

After running mp push, a PR was supposed to be excluded from the batch updates was closed in the GitHub UI. Then mp merge was run. The closed PR was merged anyway. This seems like undesirable behavior.

mp merge causes the following error

DELETE https://api.github.com/repos/org/repo/git/refs/heads/my-branch: 422 Reference does not exist []

All of our repos have the Automatically delete head branches setting checked for when PRs are merged, which may be the culprit

On GitLab enterprise, mp merge currently merges if it is possible to merge. If a repo is not configured with any merge rules requiring an approval, this means merges will occur potentially with no review. I would like the option to merge only if an explicit approval was given by a reviewer regardless of how the project is configured.

In addition, mp status should independently report the approval status of a pending MR.

It seems that the list of repository returned by init in 0.0.27 and later is much shorter than before.
E.g.:

$ ~/Downloads/mp-0.0.26-linux-amd64 init --repo-search "org:clever language:Go"
ARCHIVED-baseworker-go
ARCHIVED-oplog-replay
amazon-kinesis-client-go
analytics-latency-config-service
analytics-monitor
analytics-util
atlas-api-client
circle-v2-migrate
clever-cli
clever-go
configure
consul-service-pulse
csvutil
ddb-to-es
ecs-task-metadata-exporter
flarebot
go-bench
go-redis-reservation
go-utils
go-validation
http-science
kayvee-go
kinesis-alerts-consumer
launch-gen
leakybucket
marathon-stats
mesos-visualizer
mgohttp
microplane
mongo-lock-go
mongo-to-s3
moredis
optimus
pathio
pickabot
resolve-ip
s3-to-redshift
sfncli
sphinx
stealth
swagger-api
talks
unique-port
wag
workflow-manager

Versus

$ ~/Downloads/mp-0.0.27-linux-amd64 init --repo-search "org:clever language:Go"
circle-v2-migrate
ddb-to-es

Maybe I missed an intentional breaking change in how --repo-search works?

Hey,

Just trying MP, and noticed a bug, if two workflows are configured on a branch, and one is successful and one is skipped (because it doesn't apply in this case), when trying to merge you get this error:

merge error: Build status was not 'success', instead was 'pending'. Use --ignore-build-status to override this check.

Presumably it's treating the skipped check as pending and so I have to ignore the build status to continue the merge.

Thanks

Dependabot couldn't find a Gopkg.toml for this project.

Dependabot requires a Gopkg.toml to evaluate your project's current Go dependencies. It had expected to find one at the path: /Gopkg.toml.

If this isn't a Go project, or if it is a library, you may wish to disable updates for it from within Dependabot.

View the update logs.

reproduction steps: mp -h
Expected result: No side-effect
Actual result: Side-effect. A folder "mp" is created.

➜  a git:(master) ✗ ls -la
total 8
drwxr-xr-x 2 caleb caleb 4096 Mar 26 20:30 .
drwxr-xr-x 6 caleb caleb 4096 Mar 26 20:28 ..
➜  a git:(master) ✗ mp -h
Microplane makes git changes across many repos

Usage:
  mp [command]

Available Commands:
  clone       Clone all repos targeted by init
  docs        Generates markdown docs for each command
  help        Help about any command
  init        Initialize a microplane workflow
  merge       Merge pushed changes
  plan        Plan changes by running a command against cloned repos
  push        Push planned changes
  status      Status shows a workflow's progress
  sync        Sync workflow status with remote repo
  version     Print the current microplane version

Flags:
  -h, --help          help for mp
  -r, --repo string   single repo to operate on

Use "mp [command] --help" for more information about a command.
➜  a git:(master) ✗ ls
mp

version: latest.
OS: Ubuntu 20 on WSL2

I see there is a PR to allow empty commits during the plan phase, but when I try it out, it says unknown flag: --allow-empty-commit. This is the command I ran:

mp plan -b $BRANCH_NAME --allow-empty-commit -m "Commit msg"

BTW I installed microplane from homebrew on my Intel Mac

Hey y'all! Was wondering what the recommended approach to updating a pull request would be. I think the workflows that might want to be supported are

1. Re-init search results with updated repository list

2. Re-plan changes on latest main branch of repositories

3. Re-plan with updated command

   1. This should also support repositories which have already merged an original plan. In other words, the order of operations that one might have are

      1. Use microplane init, plan, push
      2. Get the PR merged
      3. Realize the plan was incomplete, rerun mp plan with new updated command
      4. Create new PR with new changes

   I think right now I think this results in

   2022/11/18 12:18:34 foo/bar - already merged
   

How much of that already Just Works? If someone points me in the direction, would love to contribute what I can to make the gaps filled :)

Hey y'all, my company has been using microplane with great success over the past couple of years and it's an amazing tool. However, due to business realities, the tool doesn't cover all of our use cases - while keeping the main branch up to date is awesome, we have a number of patch and beta branches that cannot have the main branch merged but also need to be maintained. The tool's current limitations mean that we have to go in an manually make changes to 20-30 branches manually which is less than ideal for changes that could be automated. Is it possible to modify the tool such that you can specify a branch, either in a flag or the text file (e.g. org/repo@branch)?

Thanks!

This is one of my favorite tools. Thank you very much for opening the issues tab.

It would be nice when using

mp init "<org>" --all-repos

to have a flag for --skip-archived so we can skip initializing archived repos.

From what I read here my understanding is that mp plan should result in a preview showing the git diff. However, that is not the case.

➜ mp init 'repo:clever/microplane' --repo-search
➜ mp plan -r microplane -b go1.9 -m "Golang 1.9 upgrade" -- sed -i 's/1.17/1.18/g' .circleci/config.yml
2022/03/26 21:29:50 planning 1 repos with parallelism limit [10]
2022/03/26 21:29:50 planning: Clever/microplane
➜ 

Interestingly the change is still made and the git diff is still stored in plan.json, it just doesn't show it for some reason:

➜  cat mp/microplane/plan/plan.json
{
    "Success": true,
    "PlanDir": "/home/caleb/misc/candel/b/mp/microplane/plan/planned",
    "GitDiff": "\u001b[1mdiff --git a/.circleci/config.yml b/.circleci/config.yml\u001b[m\n\u001b[1mindex 17e4c9f..00a8c00 100644\u001b[m\n\u001b[1m--- a/.circleci/config.yml\u001b[m\n\u001b[1m+++ b/.circleci/config.yml\u001b[m\n\u001b[36m@@ -3,7 +3,7 @@\u001b[m \u001b[mjobs:\u001b[m\n   build:\u001b[m\n     working_directory: /go/src/github.com/Clever/microplane\u001b[m\n     docker:\u001b[m\n\u001b[31m-    - image: circleci/golang:1.17-buster\u001b[m\n\u001b[32m+\u001b[m\u001b[32m    - image: circleci/golang:1.18-buster\u001b[m\n     environment:\u001b[m\n       CIRCLE_ARTIFACTS: /tmp/circleci-artifacts\u001b[m\n       CIRCLE_TEST_REPORTS: /tmp/circleci-test-results\u001b[m\n",
    "CommitMessage": "Golang 1.9 upgrade",
    "BranchName": "go1.9"
}%
➜  head mp/microplane/plan/planned/.circleci/config.yml
version: 2
jobs:
  build:
    working_directory: /go/src/github.com/Clever/microplane
    docker:
    - image: circleci/golang:1.18-buster
    environment:
      CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
      CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
    steps:

The preview was the reason I decided to try this out over shepherd. It's a really good idea so hopefully this is a easy fix.

The version in homebrew is currently updated adhoc by others.

Perhaps we can automate these updates via CI, or at minimum document how to update the brew formula in our "releasing" docs.

Homebrew/homebrew-core#77350

p.s. @Moser-ss thanks for making a PR to update the version in Homebrew!

I've got a few repositories I'm thinking about making public. These are typical workloads that I use microplane with. Stuff like normalizing package.json in Typescript/JS repositories. I noticed recently Octoherd (which also is a potential project to link to, re #104), their asking contributors to tag their repositories with octoherd which they link to.

See: https://github.com/topics/octoherd-script

Maybe we can use |#microplane](https://github.com/topics/microplane) or #microplane-script. I can coordinate adding a few projects to these topics and open a pull request to add the topic reference to the README.

What do you think?

Currently plane is doing "git diff --color HEAD^ HEAD", adding parameters like 1mand , 31m at the beginning of each line, but the Diffparser doesn't know how to parse this, and the status will not show the changes.

Faster solution: remove the "--color" param in it.

Some projects have an alternative projects section.

What do you folks think of linking to newcomer turbolift?

Even better would be to document the differences or pros and cons between the two.

mp init "-.github user:me filename:CODEOWNERS" returns

Error: unknown shorthand flag: '.' in -.github user:me filename:CODEOWNERS

while mp init "user:me -.github filename:CODEOWNERS" works fine.

(This may be a cobra thing, not an mp thing)

Dependabot couldn't find a Gopkg.toml for this project.

Dependabot requires a Gopkg.toml to evaluate your project's current Go dependencies. It had expected to find one at the path: /Gopkg.toml.

If this isn't a Go project, or if it is a library, you may wish to disable updates for it from within Dependabot.

View the update logs.

It would be very helpful to add an example to the mp push documentation on how to add multiple labels 🙏

When I am syncing more than 60 repositories I am getting this error message

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1375f36]

goroutine 31 [running]:
github.com/Clever/microplane/sync.GithubSyncPush(0x14d88f0, 0xc0000ba008, 0xc0000b8360, 0x12, 0xc0000bbf00, 0x9, 0x0, 0x0, 0xc0000bbf0a, 0x6, ...)
	github.com/Clever/microplane/sync/syncGithub.go:40 +0x316
github.com/Clever/microplane/cmd.syncPush(0xc0000b8360, 0x12, 0xc0000bbf00, 0x9, 0x0, 0x0, 0xc0000bbf0a, 0x6, 0x0, 0x0, ...)
	github.com/Clever/microplane/cmd/sync.go:87 +0x385
github.com/Clever/microplane/cmd.syncOneRepo(0xc0000b8360, 0x12, 0xc0000bbf00, 0x9, 0x0, 0x0, 0xc0000bbf0a, 0x6, 0x0, 0x0, ...)
	github.com/Clever/microplane/cmd/sync.go:58 +0x278
github.com/Clever/microplane/cmd.parallelizeLimited.func1(0xc0000965f0, 0x14d88f0, 0xc0000ba008, 0xc0000a5f80, 0x1476c08, 0xc0000b8360, 0x12, 0xc0000bbf00, 0x9, 0x0, ...)
	github.com/Clever/microplane/cmd/helpers.go:48 +0x111
created by github.com/Clever/microplane/cmd.parallelizeLimited
	github.com/Clever/microplane/cmd/helpers.go:43 +0x198 

For better or worse, my organization has projects (repos) with spaces in their names. When I attempt to push, these fail with a 404. The mp status output looks like:

Test Project				planned		(push error) GET https://gitlab.example.com/api/v4/projects/foo/Test Project: 404 {message: 404 Project Not Found}

I suspect a URL is being improperly generated without escaping special characters. I can't tell if this is a microplane issue or an issue with the gitlab library.

Running the command specified in the readme in go version go1.16.5 linux/amd64:

go get github.com/clever/microplane/cmd

yields the following error message:

go: downloading github.com/clever/microplane v0.0.32
go get: github.com/clever/[email protected] updating to
        github.com/clever/[email protected]: parsing go.mod:
        module declares its path as: github.com/Clever/microplane
                but was required as: github.com/clever/microplane

When using mp push -a dnesting with an enterprise GitLab, merge requests are created with no assignee, despite having an assignee in mp status output.