Need a verify script / test to confirm that new vendored dependencies have acceptable OSS licenses about sig-release

[justaugustus]

@philips --
@cblecker and I are planning to braindump on this and I'll take it over.

/assign
/cc @nikhita

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

[bgrant0607]

/remove-lifecycle rotten

This is relevant right now:
kubernetes/steering#21

[cblecker]

@kubernetes/sig-contributor-experience-feature-requests @grodrigues3 @spiffxp @parispittman @Phillels
This might be an important task for the 1.10 cycle.

[cblecker]

@mattfarina pointed out that https://fossa.io/ is a SaaS product that does this. We'd have to talk to them about pricing and such (we don't have private repos, which is how they track "developers").

@bgrant0607 @thockin Would something like this be worth looking into?

[bgrant0607]

We desperately need automated enforcement.
cc @caniszczyk

[tpepper]

@spiffxp poked me to get some additional ContribEx eyes on this and try to run with it in his absence...

I'm worried we're reinventing an oft reinvented wheel here.

First it would be good if we could assume a forward world where SPDX license strings and headers are standard. But in the meantime we will need to deal with outliers cleanly to build a complete list of observed licenses and pieces of code where a license was not determined. There are existing tools that do this and comprehend the umpteen gazillion different variants of whitespace and surrounding delimiting text.

In https://github.com/clearlinux/autospec/tree/master/autospec/license* files there is an actively maintained scanner with a quite comprehensive set of license file hashes observed from maintaining a full featured linux distro for a number of years. I suspect it would be nice and usable here if we were to suggest to them (and help code?) separating it out into a standalone library and github project. They even package kubernetes and for example as of 1.9.4 had determined via that automation that k8s is:
License : Apache-2.0 BSD-2-Clause BSD-2-Clause-FreeBSD BSD-3-Clause CC-BY-4.0 CC-BY-SA-4.0 CC0-1.0 CDDL-1.0 GPL-2.0 ISC LGPL-3.0 MIT MPL-2.0-no-copyleft-exception NCSA

But there are also many many more similar tools. LF has for many years had a whole annual conference on license compliance topics and there's a tonne of tooling out there.

And then there's the whole question of what is the set of compatible licenses for the k8s project. @caniszczyk is there a CNCF determination documented on this?

[thockin]

@tpepper that seems like a cool tool that could be a great basis for the checker. As it is, we have bespoke tools that extract a best-guess at license info, and a slim list of reviewers who verify changes thereto.

https://github.com/kubernetes/kubernetes/blob/master/hack/update-godep-licenses.sh

https://github.com/kubernetes/kubernetes/blob/master/Godeps/LICENSES

If we had a tool, we could maybe simplify this file to be a list of license names, or at least augment the raw file with the detected name for easier human review.

Are you volunteering?

[tpepper]

I’ve never used FOSSA so if that’s the CNCF preference I’ll defer to those who do have experience there. Otherwise I’m interested in contributing to making this layer of automation better.

[tpepper]

(I note FOSSA as the thread in pr 62088 has revived and appears headed that way at the moment)

[mattfarina]

Note, in addition to Fossa there is also https://www.fossology.org/

[cblecker]

Gonna take a trial run of FOSSA, based on the recommendation of @caniszczyk. Reached out to them here: https://groups.google.com/d/msg/kubernetes-sig-contribex/kpoYAyVUlew/4bTwVjTOBQAJ

/assign

[philips]

any updates on your FOSSA research @cblecker ?

[cblecker]

We met with FOSSA last month. The Github integration and periodic reports look decent, but the key was trying it out in our org and workflow.

My plate ended up overflowing with other tasks though, and I haven't got to setting it up and trialing it.

[philips]

cc @justaugustus

[justaugustus]

/unassign @cblecker

[justaugustus]

Email update to steering + sig-release + sig-contribex: https://groups.google.com/d/msg/kubernetes-sig-release/6oljCwkD6HQ/sH8W-uwwAAAJ

[nikhita]

/sig release
/remove-sig contributor-experience

[justaugustus]

/milestone v1.18

[justaugustus]

/area licensing
/milestone v1.18
/priority important-longterm
/kind feature

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.