kubernetes / metrics

Kubernetes metrics-related API types and clients

Kubernetes (K8s)

---

Kubernetes, also known as K8s, is an open source system for managing containerized applications across multiple hosts. It provides basic mechanisms for the deployment, maintenance, and scaling of applications.

Kubernetes builds upon a decade and a half of experience at Google running production workloads at scale using a system called Borg, combined with best-of-breed ideas and practices from the community.

Kubernetes is hosted by the Cloud Native Computing Foundation (CNCF). If your company wants to help shape the evolution of technologies that are container-packaged, dynamically scheduled, and microservices-oriented, consider joining the CNCF. For details about who's involved and how Kubernetes plays a role, read the CNCF announcement.

---

To start using K8s

See our documentation on kubernetes.io.

Take a free course on Scalable Microservices with Kubernetes.

To use Kubernetes code as a library in other applications, see the list of published components. Use of the k8s.io/kubernetes module or k8s.io/kubernetes/... packages as libraries is not supported.

To start developing K8s

The community repository hosts all information about building Kubernetes from source, how to contribute code and documentation, who to contact about what, etc.

If you want to build Kubernetes right away there are two options:

You have a working Go environment.

git clone https://github.com/kubernetes/kubernetes
cd kubernetes
make

You have a working Docker environment.

git clone https://github.com/kubernetes/kubernetes
cd kubernetes
make quick-release

For the full story, head over to the developer's documentation.

Support

If you need support, start with the troubleshooting guide, and work your way through the process that we've outlined.

That said, if you have questions, reach out to us one way or another.

Community Meetings

The Calendar has the list of all the meetings in the Kubernetes community in a single location.

Adopters

The User Case Studies website has real-world use cases of organizations across industries that are deploying/migrating to Kubernetes.

Governance

Kubernetes project is governed by a framework of principles, values, policies and processes to help our community and constituents towards our shared goals.

The Kubernetes Community is the launching point for learning about how we organize ourselves.

The Kubernetes Steering community repo is used by the Kubernetes Steering Committee, which oversees governance of the Kubernetes project.

Roadmap

The Kubernetes Enhancements repo provides information about Kubernetes releases, as well as feature tracking and backlogs.

metrics's Issues

Add pod/node labels

I'm using metrics-server to keep track of certain CPU/memory usage, but I'd like to aggregate on pod labels.

I asked this over here kubernetes-sigs/metrics-server#347 and was told that I should ask it here.

Need a document on how to use this for custom metrics api server

Hi

We are actually planning to implement this custom metrics API server on our cluster and use it for pod auto scaling. So, looking forward to an introductory document or a blog post on how to use this project.

Thanks.

Support mutli-cluster for HPA

Multi-cluster is a reasonable scenario in a production environment，but now，can not support it，can you support mutli-cluster for HPA ？

some problems about heapster

Heapster has been archived. But I used heapster in my project ,and my k8s cluster upgrade 1.14 ,now l can't get any data from heapster. someone can tell me what can i do if i wnat to use heapster in 1.14

question: context is building custom controller for `NodeMetricsList` kind

wanted to ask how should I build a custom controller using kubebuilder
for watching this resource

metrics/pkg/apis/metrics/v1beta1/types.go

Lines 50 to 58 in 538a35d

	type NodeMetricsList struct {
	metav1.TypeMeta `json:",inline"`
	// Standard list metadata.
	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
	metav1.ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
	// List of node metrics.
	Items []NodeMetrics `json:"items" protobuf:"bytes,2,rep,name=items"`
	}

here are my kubbuilder commands which failed

kubebuilder init --domain kubesimplify.com --repo kubesimplify.com/ksctl
kubebuilder create api --group metrics.k8s.io --version v1beta1 --kind NodeMetricsList

but it is failing
code

func (r *NodeMetricsListReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	_ = log.FromContext(ctx)

	// TODO(user): your logic here

	fmt.Println("TRIGGERED CONTROLLER!!")

	return ctrl.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *NodeMetricsListReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		// Uncomment the following line adding a pointer to an instance of the controlled resource as an argument
		For(&metricsServer.NodeMetrics{}).
		Complete(r)
}

Linter error

1. level=warning msg="[runner] Can't run linter goanalysis_metalinter: buildir: failed to load package cache: could not load export data: no export data for \"sigs.k8s.io/controller-runtime/pkg/cache\""
   level=error msg="Running error: 1 error occurred:\n\t* can't run linter goanalysis_metalinter: buildir: failed to load package cache: could not load export data: no export data for \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\n"

terminal

➜  ksctl-controller git:(main) ✗ make run
/home/dipankar/Onedrive/gitlab/go-playground/ksctl-controller/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/dipankar/Onedrive/gitlab/go-playground/ksctl-controller/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
# sigs.k8s.io/controller-runtime/pkg/cache
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:308:9: cannot use handles (variable of type map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration) as "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration value in return statement: map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:321:9: cannot use handles (variable of type map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration) as "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration value in return statement: map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
../../../../go/pkg/mod/sigs.k8s.io/[email protected]/pkg/cache/multi_namespace_cache.go:326:17: impossible type assertion: h.(map[string]toolscache.ResourceEventHandlerRegistration)
	map[string]"k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration does not implement "k8s.io/client-go/tools/cache".ResourceEventHandlerRegistration (missing method HasSynced)
make: *** [Makefile:55: vet] Error 1

MarshalToSizedBuffer undefined (type *resource.Quantity has no field

Getting below error while running metrics program.

var kubeconfig, master string //empty, assuming inClusterConfig
config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
if err != nil {
	panic(err)
}

mc, err := metrics.NewForConfig(config)
if err != nil {
	panic(err)
}

metricsList, err := mc.MetricsV1beta1().NodeMetricses().List(metav1.ListOptions{})

# k8s.io/metrics/pkg/apis/metrics/v1beta1
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:281:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:338:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:358:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:368:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:378:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:425:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:472:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:482:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:492:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:539:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1beta1/generated.pb.go:539:26: too many errors
# k8s.io/metrics/pkg/apis/metrics/v1alpha1
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:281:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:338:44: (*resource.Quantity)(&v).MarshalToSizedBuffer undefined (type *resource.Quantity has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:358:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:368:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:378:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:425:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:472:24: m.Window.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Duration has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:482:27: m.Timestamp.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".Time has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:492:28: m.ObjectMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ObjectMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:539:26: m.ListMeta.MarshalToSizedBuffer undefined (type "k8s.io/apimachinery/pkg/apis/meta/v1".ListMeta has no field or method MarshalToSizedBuffer)
../../pkg/mod/k8s.io/[email protected]/pkg/apis/metrics/v1alpha1/generated.pb.go:539:26: too many errors
make: *** [build-cma] Error 2

Length restrictions on External Metrics

It appears that using the generic label in MetricIdentifier.Selector comes with the following validation

invalid label value: "prod-redacted-queue": must be no more than 63 characters

I think this might need to be a different struct as it's possible for the values you're looking for to be longer then 63 characters, and RFC-1123 doesn't come into play when looking up metrics by key value.

Necessity of metrics server exposing metrics to kube apiserver.

As we know , kube apiserver periodically fetches status reports from kubelets, why should metrics server collect mertrics from kubelete and expose the same to kube apiserver ? Please explain.

implement the Object interface of k8s.io/apimachinery/pkg/apis/meta/v1

Plan to use kubectl get to display the metrics of custom-metrics-apiserver, but the Object object of TableRow in the k8s.io/apimachinery/pkg/apis/meta/v1beta1 package needs to be implemented in k8s.io/apimachinery/pkg/apis/meta/v1.
So we need to implement it in custom_metrics.MetricValue.
/cc @dgrisonnet

Deprecation of "github.com/gogo/protobuf"

"k8s.io/metrics" is using the deprecated "github.com/gogo/protobuf"
Is it possible to remove this deprecated module and replace it with a suitable alternative?

The container.Usage.Cpu().AsInt64

Dear community:
I use this function to get one pod of a container's CPU,Memory

`
func PrintAllResourceOfPodContainers(podContainers []v1beta1.ContainerMetrics){

for _,container := range podContainers{

    fmt.Println("container: ",container)

    cpuQuantiy    := container.Usage.Cpu().AsApproximateFloat64()

    memQuantiy    := container.Usage.Memory().AsApproximateFloat64()

    memQuantiy     = memQuantiy / Mi

    stoQuantiy,ok := container.Usage.Storage().AsInt64()

    containerName := container.Name

    if !ok{
        return 
    }
    msg := fmt.Sprintf("Container Name: %s \n Cpu usage: %f \n memory usage: %f MB\n storage usage: %d\n",containerName,cpuQuantiy,memQuantiy,stoQuantiy)
    fmt.Println(msg)
}

}
`
and I get this output:

container: {free5gc-upf-container map[cpu:{{59076 -9} {<nil>} 59076n DecimalSI} memory:{{30302208 0} {<nil>} BinarySI}]}
According my best knowledge, I know 1CPU = 1000m = 1000 * 1000 n, so is the cpu:{{59076 -9}} is wrong ? I think it is -6 not -9,can anyone know what wrong with this output?

Fake metrics list only "pods" resources

While trying to make unitest for metrics and using "k8s.io/metrics/pkg/client/clientset/versioned/fake" the following was observed.

the object provided as input to NewSimpleClientset() is of type v1beta1.PodMetricsList. The addition of the object is of Kind: "PodMetrics" as this is extracted in ObjectKinds() and the schema.GroupVersionKind. At the end this results in resource

schema.GroupVersionResource{Group: "metrics.k8s.io", Version: "v1beta1", Resource: "podmetricses"}

The metrices are requested using :

metricsClient.MetricsV1beta1().PodMetricses(namespace).List(getContext(options), getK8sRequestOptions(options))

However when getting the pod metrics the podmetricsesResource is predefined as Resource: "pods" thus no data found when requesting the pod metricses

var podmetricsesResource = v1beta1.SchemeGroupVersion.WithResource("pods")

// Get takes name of the podMetrics, and returns the corresponding podMetrics object, and an error if there is any.
func (c *FakePodMetricses) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.PodMetrics, err error) {
	obj, err := c.Fake.
		Invokes(testing.NewGetAction(podmetricsesResource, c.ns, name), &v1beta1.PodMetrics{})

What type of object should be provided in NewSimpleClientset for v1beta1 version to store the kind as Resource: "pods" for podmetricsesResource in fakepodmetrics.go. Is this functionality correct?

Create a SECURITY_CONTACTS file.

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md