How to Open Source at Zalando
A guide to releasing an open-source project at Zalando, Europe's largest online fashion platform.
Table of Contents
- Why Open Source?
- Our Open Source First Principles
- Project Design
- Where to Publish: Main Org, Incubator, or GHE?
- Project Basics
- Code Review
- Creating a README
- Official Namespace
- Applying Changes
- Versioning
- Maintainers
- Stay Secure
- About Licensing
- Quick FAQ
- Which License Do We Use?
- Must I Use MIT?
- I don't like the MIT license. Can I use another license?
- What if I fork an external project and/or contribute to it?
- I'm open-sourcing a library. What should I do?
- What if my team uses an external project whose license is not Zalando-recommended?
- Who is the license two-factor?
- Does my project need a LICENSE file?
- Do I need to add a license header to every source file?
- What about a README note?
- I still have licensing questions. What can I do?
- But I am a delivery lead.
- But I am a department head.
- Repository of Meta Information
- Restrictions Imposed by the License
- There Is No License
- Quick FAQ
- Other Repository Information
- Working With External Contributors
- Make Forks Meaningful
- Deprecate Responsibly
- Project Promotion
- Contributing to Non-Zalando Open-Source Projects - General - Google Projects
###Why Open Source?
Because it can: improve quality, mitigate risk, increase trust, save us money, expand our technology choices, be fun, enable us to give back to the community, strengthen our tech brand, and attract talent.
###Our Open Source First Principles
Vision: We strongly believe that open source software benefits the tech community, and that providing broadly useful code to the world is a virtue. We strive to work in an open source way to the betterment of Zalando and the world.
- Do “Open Source First”: If your Zalando project can also be useful to non-Zalandos, release it as open source from the start.
- Take Ownership: Your team is responsible for ensuring that it’s possible to open-source your project. Your delivery lead is available for guidance.
- Share Your Code: All code shared between teams must be open source.
- Be Safe: To ensure the broadest possible use of your project, use the MIT License only.
- Deliver Quality: Provide a great out-of-the-box experience.
- Provide Documentation: Include a clear README and default working configuration.
- Stay Secure: Make sure your project doesn’t include Zalando specifics, such as credentials and private identifiers.
- Ask for Help: Find colleagues to brainstorm ideas for your project and to review your work.
- Promote: Tell the world about your project via blog posts, social media and conference talks.
- Join the Open Source Guild: Help us make open source stronger at Zalando! Come on, it'll be fun. :)
This blog post might also be of interest.
###Project Design
####How to Build a Useful Project (Checklist) Use this checklist to develop your "product mindset" for your project and make is as useful as possible.
####Never open-source these:
- PCI DSS-related projects: e.g. payment services
- Domain knowledge
- Anything that would risk our competitive advantage:
- confidential source code
- recommendation algorithms
- Customer data
- Unique Selling Points (USPs)
If you're open-sourcing a project that has contained sensitive information in the past, the sensitive information can still be retrieved from the Git commit history. Create an entirely new Git repo for it before pushing it to GitHub.
No issues? Great! On to the next section ...
###Where to Publish: Main Org, Incubator, or GHE?
Based on quality, usefulness and maintenance considerations, we use this matrix to decide how to classify and place our projects:
- InnerSource = GitHub Enterprise
- Coding in the Open = Incubator
- Open source = main Zalando org
In the inverse, you could think of these three categories as “steps” for your project as it develops from an idea to a minimum viable product (MVP) that someone on the outside could understand, use and possibly work on as a contributor.
The next sections offer more details on differences between open source, “coding in the open” and InnerSource.
####What Makes a Project “Open Source” An open source project:
-
is useful beyond Zalando. It is free of Zalando dependencies and simple for a non-Zalando to install and start using.
-
has high-quality documentation that is up-to-date and clear about what the project does. (This template can help you.)
-
is tested. It has automated tests and takes advantage of test coverage.
-
is under active development, or is stable enough to be considered a “finished” product. If the project is incomplete, at least one maintainer has worked on it in the last three months. If it’s stable and doesn’t require constant maintenance, you’ve stated as much in your README.
-
is innovative. If it duplicates an existing project, it does at least one thing better, faster, differently, etc., or is higher-quality.
-
meets our non-negotiable guidelines regarding security and compliance.
-
is an MVP. It either meets or surpasses “minimum viable product” status. An outside developer could use it and even contribute to it. If it’s buggy or very early-stage, it includes a brief development status in the intro stating as much. (This template can help you.)
-
has a plan. Its maintainers care about making it a success. They commit to responding to PRs and issues in a timely manner (48-72 hours), thank contributors, and convert quality contributors to trusted maintainers as appropriate.
-
Example projects: Patroni, Connexion, Zappr, Python NSEnter, Tailor, Spoton.
-
Where it belongs: Zalando main org
We have also published some document-only repositories like our RESTful API guidelines, our “How to Open Source” guidelines, the Tech Radar, and our Rules of Play. These reflect our organizational perspective on technical topics, and are therefore of potential interest to other companies who are looking for ideas and case studies. We should keep them limited in number and broad in scope.
What Makes a Project “Coding in the Open”
“Coding in the open” is how we describe a Zalando project that is already public—or that you want to make public for personal reasons—but not currently useful beyond Zalando Tech because it is affected by at least one of the following issues:
-
it is dependent on other Zalando projects
-
it is difficult or impossible for a non-Zalando to install
-
it has documentation that lacks essential user information, or lacks docs altogether
-
it isn’t currently maintained
-
it is in pre-MVP (minimum viable product) status
-
duplicates the functionality of an existing project and is of questionable quality (due to lack of code review or testing)
-
Example projects: oakkeeper, STUPS, stups2go, solution-center-login, Tarbela, Tranzlate
-
Where it belongs: Zalando Incubator
#####Quick Incubator FAQ
######Why did we make the Incubator? To support our Open Source First principle, "Share Your Code,” while reserving our main GitHub org for projects useful beyond Zalando.
######Can you launch a project directly to the Incubator? Yes. And you should, if it meets “coding in the open” criteria.
######Do Incubator projects have to follow our security and compliance guidelines? Yes, even if we are developing them for Zalandos only, “experimental,” etc.
######Can an Incubator project ever appear in the main Zalando GitHub org? Absolutely. Even if Incubator projects aren't currently useful outside of Zalando, we'd like to think that one day they might be.
######What happens when you transfer a repo from one org to another? The URL is re-routed to the new organization (zalando-incubator). All git clone, git fetch, or git push operations targeting the previous location will continue to function as if made on the new location. Links will be transferred to the new repository.
That said, GitHub strongly recommends that anyone who had a reference to the old link locally change to the new URL. Read GitHub’s documentation to learn more.
######What happens if I “code in the open” on the main org? The Guild will ask you to transfer the project yourself to the Incubator, first by filing an Issue on the project and then via HipChat/one-on-one. If we don’t get any response within 14 days, or if you agree to the transfer but don’t take action within a week (seven days), your project will be relocated to the Incubator for you.
######Is it possible to transfer GitHub issues from one project to the other? No, GitHub doesn’t allow it. (There are migration tools that can copy issues and their comments to another project, but these don't transfer the relevant author info; they assign ownership to the account used by the migration tool.)
######Is it possible to transfer the ownership of a Github repo to someone else? Yes.
####What Makes a Project InnerSource InnerSource projects meet all the same criteria as “coding in the open” projects, but with one major difference: they appear on GitHub Enterprise, and are not accessible to the public. No one outside of Zalando Tech can see or contribute to them.
Wait, I thought we were “Open Source First.” Doesn’t keeping repos on GHE contradict that?
Nope. Not every project we create will be appropriate for sharing publicly. Some projects will be too sensitive for publication. Other projects would act as “noise,” because they are too tightly coupled to what we do internally. An organization's open source footprint says a lot about that organization, especially if the org cannot maintain a good signal-to-noise ratio.
However, we still want you to share these projects inside Zalando. This is why we advocate the InnerSource collaboration model.
Put simply, InnerSource operates just like open-source in that project teams invite, accept and reject PRs; provide quality documentation; and build them gradually. The main difference is that InnerSource is limited to a single organization—in our case, Zalando Tech.
With InnerSource, we encourage you to make your GitHub Enterprise organization open to other internal teams so they can find out about your work, fix bugs, make PRs, and even add features that your own team currently has no time to develop.
We request that you use private GHE repositories only if they include sensitive information that can't store elsewhere.
Where InnerSource projects belong: GitHub Enterprise
##Project Basics
###Code Review
Ask your team and other peers to:
- review your code
- install and
- test your project prior to public release.
Not sure what to ask for, or how to peer-review? This list of 11 best practices should help.
###Creating a README
####Do:
- Use this checklist.
- Break up text often, for better readability.
- Think about SEO.
####Don't:
- Refer to Zalando specifics, such as internal teams and processes
- Include large chunks of code without explaining what they represent
- Include any code that presents security vulnerabilities
####Syntax and Formatting Markdown is the simplest and most easily understood syntax; we recommend using it for all your documentation. However, we realize that there are exceptions: PyPi, for example, uses reStructuredText, and the Python community in general doesn’t use Markdown. If Markdown isn’t practical, then we recommend using only one GitHub-supported markup format.
###Official Namespace
The official namespace for our projects is ‘org.zalando’, where applicable.
###Applying Changes
All repository changes, including those made by maintainers, should come from GitHub pull requests so that we can streamline review and change tracking (as per GitHub Flow). Everyone should have their own fork, though you can still edit READMEs/documentation/related files with the GitHub “Edit” button. The ‘master’ branch should be the accepted development head; pull requests get merged there.
###Versioning
Version all project artifacts should be semantically. Tag all versions in GitHub with the exact version name (like ‘0.1.0’; do not prefix tags with “v.” or similar). For a better user experience, use the GitHub “release notes” feature to add notes whenever you change something in the new version.
###Maintainers
Maintainers are the contact people for a project. They are also the only contributors who can package new versions and apply changes to the repository (i.e., merge pull requests). Every project must have at least one maintainer. This helper script gives us an overview of repos that users are maintainers for.
The Open Source Guild reserves the right to contact maintainers to ensure a project remains active/maintained. If the project is not being maintained, we will work with you to either find a new maintainer or remove the project from our organization page. Please be responsive to all internal queries about your project and its status.
####Create a Maintainers File
Every project needs a ‘MAINTAINERS’ file (listing all maintainers) at its root. Your build/packaging configuration file (e.g., pom.xml for Maven) can fulfill the purpose of a MAINTAINERS file. Format:
[full name] <email address>
[second full name] <email address>
etc.
Our Catwatch application will collect maintainers from the MAINTAINERS files.
####Be Prompt and Responsive
Respond promptly to pull requests and issues. “Within 72 hours” is a good window. Open issues do not make your project “look popular.” Instead, they make it look like you're neglecting your project. If project workload becomes unmanageable, ask the Guild or the community for help.
If you are away/on vacation and can’t respond to PRs/issues promptly, find someone who can.
If you're not going to accept a PR, reject it ASAP and include a brief explanation why.
####Use Issues Creatively Issues can be good for planning or for onboarding contributors. Issues should include a description of the point, question, discovery, or other detail prompting the issue.
Issues that consist solely of a title appear unprofessional and do not do much to invite discussion from the community.
Label your issues with clear tags. This is a great way to organize and categorize issues.
###Stay Secure #####Two-Factor Authentication Is Required GitHub organization members must enable Two-Factor Authentication (2FA/MFA) in keeping with our Open Source First principle to "Stay Secure." Read GitHub's post on 2FA for more information.
Don't have a smartphone, and/or want to give your phone number to GitHub? According to GitHub support, SMS or a TOTP app are currently the only primary 2FA methods that work. You can set up 2FA with a Google Voice SMS number, but should add a U2F device as backup.
###About Licensing
####Quick FAQ
#####Which license do we use? The MIT license. MIT is succinct, straightforward, and easy use in closed-source projects. It allows the most broad usage of our source code, and keeps open-sourcing easy and safe.
For documents like this how-to, we recommend using Creative Commons Attribution 4.0 International.
#####Must I use MIT? Yes, for all newly created open-source projects.
#####I don't like the MIT license. Can I use another license?
Not without a compelling reason.
#####What if I fork an external project and/or contribute to it?
Keep the original license.
#####I'm open-sourcing a library. What should I do? Consider using a weak copyleft license that won’t restrict the software that uses it to the same license; will allow usage in closed source software; and will potentially increase the number of users and contributors.
#####What if my team uses an external project whose license is not Zalando-recommended?
You can can use GPL code — but only internally. Be sure it's a version of the GPL that continues to allow for the ASP loophole. AGPL and versions of the GPL with additional restrictions won't work.
#####Who is the license owner?
Zalando SE.
#####Does my project need a LICENSE file?
Yes, at the root of the repository that contains the copy of the selected license (see above). Here is an example for MIT.
#####Do I need to add a license header to every source file?
No. In fact, we discourage it because it blows up file sizes, requires some build checking/pre-processing, and sometimes leads to situations like this. It's also not needed for the licenses we use. Having one LICENSE
file in the repository is enough.
#####What about a README note?
Yep. Every README{.md,.rst} file must state the following at the end:
The MIT License (MIT) Copyright © [yyyy] Zalando SE, https://tech.zalando.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Replace the [yyyy] field with the year that you created the project, and do not update it. Do not provide multiple years.
#####I still have licensing questions. What can I do?
Ask the Open Source Guild or your delivery lead. See also our TechWiki page (internal doc) for a link to more detailed information.
#####But I am a delivery lead.
Ask your department head.
#####But I am a department head.
Management can work with Legal to determine Intellectual Property concerns.
####Repository of Meta Information
Many package managers include a feature to make the applied license machine readable. Use these! An example for Maven:
<license>
<name>MIT</name>
<url>https://opensource.org/licenses/MIT</url>
<distribution>repo</distribution>
</license>
</licenses>
An example for Node, according to this:
“license”: “MIT”
An example for Scala (with sbt):
licenses += ("MIT", url("http://opensource.org/licenses/MIT"))
####Restrictions Imposed by the License “Dependency” typically means “being linked with,” “included in your artifact,” or “depends on it during runtime.” Dependencies can limit you. To remain in compliance, check the licenses of your projects. Your build tool’s license does not affect your software’s license. A jar file or Python dependency will affect your software.
#####Unusual Additions
As stated by Zalando Legal, it is OK to use React and other Facebook open-source software projects for Zalando projects.
####There Is No License
If there is no license statement, the author automatically receives a copyright. This implies that no one has the right to modify or redistribute the software. If you really need the software, contact the author (who is likely unaware) and ask him/her to provide a proper license.
###Other Repository Information
####JVM Artifacts Host JVM artifacts (*.jar) on Maven Central in the ‘org.zalando’ group. To do this, get a Sonatype account. If you don't have an account yet, register with Sonatype using your Zalando email address.
If you want to push to Sonatype but not to *.zalando, register a different Sonatype account with a non-Zalando email address.
####Python Packages Host Python packages on PyPI (PyPI has no namespaces) and make sure that multiple persons have “maintainer” rights.
####SBT plugins
- Publish on Bintray. An example of the publishing process is here.
####Node Modules Node modules now have namespaces. Prefix them with a short product name: e.g. karma plugins are prefixed “karma-”; the same goes for gulp, grunt, etc. Host your Node modules in the public NPM registry. Here is how to publish to NPM.
####Docker Images You can currently browse it here, or with the Pier One command line utility. We have an open source registry that everyone can read. It is deployed in the AWS open source account and Docker images can be pushed by any team member to their respective team repo:
$ # you need to login to registry-write.opensource.zalan.do (workaround for Docker V2 bug)
$ pierone login --url registry-write.opensource.zalan.do
$ docker push registry-write.opensource.zalan.do/myteam/myartifact:1.0
$ # on any other computer:
$ docker pull registry.opensource.zalan.do/myteam/myartifact:1.0 # no auth needed for download!
###Working with External Contributors
The Guild supports you in recruiting non-Zalandos to contribute to your project. We simply require that you have external contributors sign a contributor licensing agreement (CLA). A few good models to follow and adapt: .Net Foundation example (electronic submission via GitHub account); Google’s CLA for contributing to AngularJS is a simple click-through form with a Googlebot that automatically checks for signatures; Selenium/Software Freedom Conservancy uses a Google form.
###Make Forks Meaningful
“Forks are for making your own snapshot of a codebase so that you can make a new version of it with your own special sauce, or so that you can contribute a change in the form of a pull request. Simply, you must make a fork whenever you need to modify the codebase, but do not have direct access to do so. New users don’t understand this and end up equating the ‘fork’ button with ‘download’ or ‘bookmark’. Little do they know, you can download code directly from the original repository and you can bookmark things using Github’s stars.” —Eric Greer
To fork, or not to fork? Some guidelines:
- Only keep the fork open as long as needed.
- “True/Long-Lived forks” are highly discouraged, even from Zalando projects. We avoid internal forks in order to avoid diverging widely from what we have published, and the inevitable hassle of getting the project to a state where we can support both code bases
- Avoid two forks of the same project.
If your goal is to make a small fix to a project, use your own/personal GitHub account.
###Deprecate Responsibly
####How to Deprecate Add “Deprecated” at the top of your README, as well as a notice stating your plans for the project: deletion, finding a new owner, etc.
After announcing your decision to deprecate the project:
- notify your users. Put a notice on your README.
- Wait 60 days.
- Try to find a new owner. Internal options first, then seek an external owner. Ask the Guild for help.
- Try to find an external owner. More guidance on this to come soon; for now, alert the Guild of any such plans.
####Tips for Finding a New Owner Internally, you can use internal mailing lists and HipChat to announce your need. Externally, social media platforms and community boards work well. Add 1-2 sentences to your announcements suggesting how your project might have potential to evolve into something bigger and better.
###Project Promotion
####zalando.github.io: Our Open-Source Projects Dashboard All Zalando open-source projects are listed on zalando.github.io (also called CatWatch): our own metrics dashboard measuring our most popular public projects and our most active contributors in terms of commits, stars and forks. Please add a .catwatch.yaml file to the root of your repository to set a human-readable project title and image URL.
####Proactively Communicate Share your project with:
- relevant LinkedIn Groups
- community forums/boards
- e-newsletters and websites/newsletters dedicated to the problem(s) your project is trying to solve, the relevant languages, etc.
- if you have contacts at a university, pitch your project to their students
- major players in the industry
###Contributing to Non-Zalando Open-Source Projects
####General We encourage you to contribute to other open-source projects in ways that benefit Zalando — for example, by making bug fixes in Apache, or submitting a patch to a language. Let the Guild know about your external contributions so we can help you get the recognition and support you deserve.
CLAs that are safe: Oracle, Apache.
####Google Projects For typical CLAs, we are safe — but ask our legal team (guild can provide their contact info) to double-check whenever you’re in doubt.