service-mesh-performance / service-mesh-performance

Standardizing Service Mesh Value Measurement

If you’re using Service Mesh Performance or if you like the project, please ★ star this repository to show your support! 🤩

SMP Community Meetings are now merged into CNCF TAG-network Meetings on the 1st and 3rd Thursday of every month at 9 AM Pacific Time. (Meeting Minutes | Zoom)

Service Mesh Performance (SMP)

A common format for capturing and describing:

1. performance test configuration
2. service mesh configuration
3. environment configuration
4. workload configuration
5. performance test results

This specification is currently being advanced through the CNCF Service Mesh WG and the Meshery project. See a screenshot of SMP's canonical implementation.

Overview

Project Overview Presentation

Implementations

The following projects or products are compatible with / have implemented SMP.

Meshery

Meshery is the multi-service mesh management plane offering lifecycle management of more types of service meshes than any other tool available today. Meshery facilitates adopting, configuring, operating and managing performance of different service meshes and incorporates the collection and display of metrics from applications running on top of any service mesh.

See the sample results file.

---

If you’re using Service Mesh Performance or if you like the project, please ★ star this repository to show your support! 🤩

Community and Contributing

Our projects are community-built and welcome collaboration. 👍 Be sure to see the Layer5 Newcomers' Guide for a tour of resources available to you and jump into our Slackto engage! Contributors are expected to adhere to the CNCF Code of Conduct.

✔️ Join any or all of the weekly meetings on the community calendar.
✔️ Watch community meeting recordings.
✔️ To Access Community Drive, fill Community Member Form.
✔️ Discuss in the Community Forum.
✔️ Browse the Community Handbook.

Not sure where to start? Grab an open issue with the help-wanted label.

About Layer5

Layer5's cloud native application and infrastructure management software enables organizations to expect more from their infrastructure. We embrace developer-defined infrastructure. We empower engineer to change how they write applications, support operators in rethinking how they run modern infrastructure and enable product owners to regain full control over their product portfolio.

License

This repository and site are available as open source under the terms of the Apache 2.0 License.

service-mesh-performance's Issues

[Docs] Create smp documentation google doc.

Current State:
The Service Mesh Performance has no google doc for additional reference

Desired State:
create a reference google doc for service mesh and add to the down part of the doc README.md for additional reference. You can recreate this meshery doc https://docs.google.com/document/d/17guuaxb0xsfutBCzyj2CT6OZiFnMu9w4PzoILXhRXSo/edit

---

Contributor Resources

• Service Mesh Performance documentation site
• Service Mesh Performance documentation source
• Instructions for contributing to documentation

Add support for egress gateways to SMPS

Add service mesh version support to sm.proto

Current Behavior
sm.proto identifies service mesh by name, but excludes other metadata like service mesh version number.

Desired Behavior
SPMS will need to afford for identifying a given service mesh's version #.

[Site] Reformat the footer section

Description

Currently, the footer section houses the social icons. The section should be reformatted to include -

• A site guide
• Social buttons
• A twitter window

Inspiration can be taken from the Meshery site:

Create SMP Documentation Site Page.

Current Behavior

The contributor resource takes you to the Meshery Documentation site page -https://meshery.layer5.io/docs/

Desired Behavior

We need it to be directed to SMP site https://smp-spec.io/docs/ but there is none available. This will help complete update on documentation template. #75

---

Resources

You can copy from the Meshery https://github.com/layer5io/meshery/blob/master/docs/pages/index.html with tweaks to suit SMP

• Meshery documentation site
• Service Mesh Performance documentation source
• Instructions for contributing to documentation

[Site] Fix blog layout

Description

The blog posts on the smp-spec.io site open in a pop-up box in the same tab.

Expected Behavior

The posts should have a post layout (can be copied from the Layer5 blog) and should be set to open in a new tab.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'https://smp-spec.io/'
2. Scroll down to the 'Blog' section.
3. Click on any of the posts.
4. See error

[Site] Add test run stats

Description

Similar to the display of the number of tests run as seen on the meshery.io site, it would be good to see this same number displayed on smp-spec.io in the Implementation section, below the gif image.

[Site] Showcase github stats for the SMP repository

Description

Display the github stats for the smp repo on the smp-spec.io site, in the Implementation section, below the gf image and the test-run stats (see #93).

[Docs]Add RVM Description to SMP Documentation

Current State:
Windows users are having issues using Jekyll and there's no description to help with that.

Desired State:
Provide a description of RVM to the set up your development environment section of the SMP Doc by adding this line.

Note: Windows users can run Jekyll by following the [Windows Installation Guide](https://jekyllrb.com/docs/installation/windows/) and also install Ruby Version Manager [RVM](https://rvm.io) - a command-line tool which allows you to easily install, manage and work with multiple ruby environments on your local machine.Alternatively, if you're running Windows 10, updated to version 1903 Build 18362 or higher you can upgrade to Windows Subsystem for Linux [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10)

The additional lines can be added right after Jekyll Docs .👍🏽

---

Contributor Resources

• Service Mesh Performance documentation source
• Instructions for contributing to documentation

Create website for SMPS

Current Behavior
Today, this initiative is best represented at https://layer5.io/performance. This initiative is in need of a dedicated site to describe its goals, how to get involved, and so on.

Desired Behavior
A new site created using Jekyll and GitHub Pages.

---

Resources
See https://meshery.io as an example.

[Release] Create SMPS v0.1.0

Current State:
SMPS as a spec has progressed to (arguably past) the point of a v0.1.0.

Desired State:
A v0.1.0 release needs to be made.

[Spec] Add a traffic.proto

Current State:
Traffic protocols will be referenced in multiple places within SPMS (e.g. in the workload definition and the performance test configuration).

Desired State:
It will be good to define traffic specifications independently.

Reference:
When supporting gRPC, will the testURL be blank? Or be a function name? I think blank because it will just be a ping test.

Originally posted by @leecalcote in meshery/meshery#1156

create netlify build

Description

Create a netlify build for the smp-spec.io site, similar to the one present for the layer5.io site on the layer5 repository.

Add a protobuf file

Currently, we are using smps.yaml as a spec.
A protobuf file to the project will greatly help in identifying the type and make the spec language agnostic.

Support for tags/labels

Support of tags or labels for SMPS can turn out very useful. An very obvious and mostly the most important advantage is the ease of indexing and searching through the reports. Also considering the fact that large organizations might have stored 100s of 1000s of performance test results stored, something like tags or labels will help speed up the search by great magnitudes.

The question remains which would be better, tags or labels.

Tags :

tags: 
- v1
- prod
- project-xyz
- wrk2

Labels:

labels: 
- version: 1
- env: prod
- app: project-xyz
- tool: wrk2

In my opinion labels seem more powerful.

Two copies of the "Analyzing with SMP" blog

Description
After the merge of the change from smps to smp, there is an extra copy (a residual copy of this post)

Expected Behavior
Only the "SMP" post should remain.

Screenshot

Environment:

• OS: [e.g. Ubuntu]
• Browser: [e.g. Chrome, Safari]
• Version: [e.g. 22]
• Device: [e.g. laptop, iPhone 8]

---

[Optional] To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

[Chore] Reconcile img and assets folders; Consolidate to img folder.

Current Behavior
The img/readme and the /assets folders are duplicative (for the same purpose). These two folders need to be reconciled by deduplicating their contents.

Desired Behavior
All of these assets are consolidated into the /img/readme folder.

Enum for `service_mesh` type

To guarantee compatibility between SMPS implementations, we may want to use an enum for service_mesh.

Originally posted by @leecalcote in #31

Add appropriate project badges to this repo's README

Each of the Meshery repos include a set of project badges.

Please add the applicable set of badges to this repo's readme.

Example badges from main meshery repo:

[Docs] Create contributing docs for the smps-spec.io site

Current State:
Site is created by Jekyll.
Site runs on GitHub Pages.
Site sits in this repo under /docs.

Desired State:
Contributing docs are needed here. The same docs used for either layer5.io or meshery.io can be copied.

---

Contributor Resources

• Meshery documentation site
• Meshery documentation source
• Instructions for contributing to documentation

Add welcome message for first time contributors

Current Situation
New contributors to this repo should be celebrated. A portion of this celebration can be automated.

Desired Situation
We can use a GitHub bot to do this. The config.yml, welcome folder and welcome image can be copied (and updated) from the the Meshery repo and placed into the .github folder in this repo.

Add sample client-side test result

The client-side test result should contain throughput, errors, latency and so on from a performance test.

Add Stale bot to this repo

Description
Unaddressed issues that remain open for long periods may be automatically marked stale and closed.

Implementation
Copy stale.yml from the Meshery repo and put into the .github folder here.

Add a sample smps.yaml under examples folder

Current Behavior

We don't have a sample file to let know how our specification looks like.

Desired Behavior

Add a sample smps.yaml under examples folder

Spec: Remove auto-generated go-client files from the repository

Current Behavior

We are storing generate golang files which in our repository which should not be there.

Desired Behavior

Implement a CI script to generate them every time after a push or have a custom bash-script to do so.

---

[CI] Add releasedrafter to this repo

Description
Leverage (copy) the releasedrafter configuration from the meshery repo.

• https://github.com/layer5io/meshery/blob/master/.github/workflows/release-drafter.yml

Integrate traffic.proto spec into workload proto

Eventually, we'll want this to refer to the traffic.proto - #38

And this, too - https://github.com/layer5io/service-mesh-performance-specification/blob/master/smps.proto#L13

Originally posted by @leecalcote in #41

CI : Introduce Prototool for the protobuf

Current Behavior

We don't have a CI nor a linter to maintain & check consistency among the protobuf file.

Desired Behavior

Consider addition of prototool as a CI check of the repository.

---

Resources

• https://github.com/uber/prototool#configuration
• https://buf.build/docs/migration-prototool
• https://github.com/bufbuild/buf

Update "Other Meshery resources" README section

Update "Other Meshery resources" README section with current community resource information. See Meshery repo for those details.

Change project name from SMPS to SMP

Current State:
SMP is shorter and gets people less tripped up. It doesn't need "Specification" in the title. Many people will already add the word specification as a descriptor when describing SMPS, so the "S" often becomes redundant.

Desired State:

• Change this repository's name (drop "specification")
• Remove the "S" in SMPS in the logo.
• Update elsewhere....?

---

Contributor Resources

• Meshery documentation site
• Meshery documentation source
• Instructions for contributing to documentation

Replace old logo with Layer5 logo

The old logo shown in the README needs to be replaced with the Layer5 logo available here /

Add CII Badge to SMPS readme

[Docs] Remove extraneous published URLs

Current State:
The site's sitemap has a few pages unnecessarily published.

Desired State:
Non-essential, template, or test pages should not be public.

---

Contributor Resources

• Meshery documentation site
• Meshery documentation source
• Instructions for contributing to documentation

Remove unused particle.js (has security vulnerabilities)

Add Blog Post Author Name to posts.html layout

Canonical implementation example animated gif missing

The service mesh performance specification example.gif went missing recently.

Delete old blog post that should have been replaced

Description

• An old blog post was updated but not replaced in https://smp-spec.io/

Expected Behavior

• The old post should be removed.

Screenshots

Create Issue Templates

This repository would benefit from the same set of issue templates found in the other Layer5 repos.

@gunjanmimo or @ashiscs, would you mind assisting here?

Create a fifth specification to separate out workload metadata

Current Behavior
The specification nominally considers for the description and all of the interesting metadata of the application (e.g. application name, application version, helm chart version #, and so on).

Desired Behavior
The definition of the workload profile definition should be expanded on.

Consider definitions like CNAB, Docker Compose, Helm, and so on.

[Site] Fix the Contributors section

Description

The logos listed in the Contributors section currently occupy two lines.

Expected Behavior

They should be adjusted onto the same line, either by adjusting the logo sizes or more preferably by adding a horizontal scroll to the section.

Screenshots

Environment:

• OS: Windows 10
• Browser: Firefox
• Device: Laptop

---

[Optional] To Reproduce
Steps to reproduce the behavior:

1. Go to https://smp-spec.io/
2. Click on Contributors in the header
3. See error

[Site] Fix header size

Description

Reduce the header length to make it more compact and accordingly resize the smp logo to fit the header. This will take some attempts at playing around with different sizes for the header and the smp logo to achieve the perfect combination.

Steps to reproduce

1. Go to https://smp-spec.io/
2. Observe the header

Update service mesh performance specification result.yaml

Currently examples/service mesh performance specification result.yaml isn't following smps.yaml format. This needs to be updated.

Add Reminder in the Welcome Bot to Star SMP Project

Current Behavior
A welcome bot has been configured to usher in contributors new to this project. However, a reminder to star the project is missing in the message.

Desired Behavior
The config.yml should be updated, you can take a cue from the highlighted line 22

---

Resources

Alternatives / Additional Context

Update Issue Templates to reflect that of SMPS

Description
The issue templates in this repo refer to Meshery.

Expected Behavior
These templates should refer to the contribution process specific to SMPS that @PluckyPrecious did such a great job of creating.

[Site] Remove extra padding

Description
The Resources and Connect sections have unnecessary top and bottom padding. Remove the extra padding present in these sections and adjust the existing content accordingly to make the sections look more presentable.

Screenshots

Remove smps.yaml from this repo

Description
The smps.yaml has been displaced by smps.proto and sm.proto.

Expected Behavior
smps.yaml should be deleted.

Add example performance test result screenshot

Add this screenshot to the readme.

Add standard set of labels to this repo

Current Behavior
Please add the ~50 labels to this repo.

[site] Add `image` to default layout page's front matter

Current Behavior
When the site link is shared, no image is retrieved and displayed.

Desired Behavior
The site should specify an image in the metadata. jekyll-seo can be used to assist here.

The amps-logo should be used as this image.

Separation of specification concerns into individual files.

Current Behavior
The single spec is long, making it hard to reason over and making use cases involving only the performance test configuration more difficult than they need to be.

Desired Behavior
Each of the five areas of the specification should be broken into individual specification files.