Coder Social home page Coder Social logo

dotnet-kube-client's Introduction

KubeClient

Build Status (Travis CI)

KubeClient is an extensible Kubernetes API client for .NET Core (targets netstandard1.4).

Note - there is also an official .NET client for Kubernetes (both clients actually share code in a couple of places). These two clients are philosophically-different (from a design perspective) but either can be bent to fit your needs. For more information about how KubeClient differs from the official client, see the section below on extensibility.

Prerequisites

Note: If you need WebSocket / exec you'll need to target netcoreapp2.1.

Packages

  • KubeClient (netstandard1.4 or newer)
    The main client and models.
    KubeClient

  • KubeClient.Extensions.Configuration (netstandard2.0 or newer)
    Support for sourcing Microsoft.Extensions.Configuration data from Kubernetes Secrets and ConfigMaps.
    KubeClient.Extensions.KubeConfig

  • KubeClient.Extensions.DependencyInjection (netstandard2.0 or newer)
    Dependency-injection support.
    KubeClient.Extensions.KubeConfig

  • KubeClient.Extensions.KubeConfig (netstandard1.4 or newer)
    Support for loading and parsing configuration from ~/.kube/config.
    KubeClient.Extensions.KubeConfig

  • KubeClient.Extensions.WebSockets (netstandard2.1 or newer)
    Support for multiplexed WebSocket connections used by Kubernetes APIs (such as exec).
    This package also extends resource clients to add support for those APIs.

    Note that, due to a dependency on the new managed WebSockets implementation in .NET Core, this package targets netcoreapp2.1 (which requires SDK version 2.1.300 or newer) and therefore only works on .NET Core 2.1 or newer (it won't work on the full .NET Framework / UWP / Xamarin until they support netstandard2.1).
    KubeClient.Extensions.WebSockets

If you want to use the latest (development) builds of KubeClient, add the following feed to NuGet.config: https://www.myget.org/F/dotnet-kube-client/api/v3/index.json

Usage

The client can be used directly or injected via Microsoft.Extensions.DependencyInjection.

Use the client directly

The simplest way to create a client is to call KubeApiClient.Create(). There are overloads if you want to provide an access token, client certificate, or customise validation of the server's certificate:

// Assumes you're using "kubectl proxy", and no authentication is required.
KubeApiClient client = KubeApiClient.Create("http://localhost:8001");

PodListV1 pods = await client.PodsV1().List(
    labelSelector: "k8s-app=my-app"
);

For more flexible configuration, use the overload that takes KubeClientOptions:

KubeApiClient client = KubeApiClient.Create(new KubeClientOptions
{
    ApiEndPoint = new Uri("http://localhost:8001"),
    AuthStrategy = KubeAuthStrategy.BearerToken,
    AccessToken = "my-access-token",
    AllowInsecure = true // Don't validate server certificate
});

You can enable logging of requests and responses by passing an ILoggerFactory to KubeApiClient.Create() or KubeClientOptions.LoggerFactory:

ILoggerFactory loggers = new LoggerFactory();
loggers.AddConsole();

KubeApiClient client = KubeApiClient.Create("http://localhost:8001", loggers);

Configure the client from ~/.kube/config

using KubeClient.Extensions.KubeConfig;

KubeClientOptions clientOptions = K8sConfig.Load(kubeConfigFile).ToKubeClientOptions(
    kubeContextName: "my-cluster",
    defaultKubeNamespace: "kube-system"
);

KubeApiClient client = KubeApiClient.Create(clientOptions);

Make the client available for dependency injection

The client can be configured for dependency injection in a variety of ways.

To use a fixed set of options for the client, use the overload of AddKubeClient() that takes KubeClientoptions:

void ConfigureServices(IServiceCollection services)
{
    services.AddKubeClient(new KubeClientOptions
    {
        ApiEndPoint = new Uri("http://localhost:8001"),
        AuthStrategy = KubeAuthStrategy.BearerToken,
        AccessToken = "my-access-token",
        AllowInsecure = true // Don't validate server certificate
    });
}

To add a named instance of the client:

void ConfigureServices(IServiceCollection services)
{
    services.AddNamedKubeClients();
    services.AddKubeClientOptions("my-cluster", clientOptions =>
    {
        clientOptions.ApiEndPoint = new Uri("http://localhost:8001");
        clientOptions.AuthStrategy = KubeAuthStrategy.BearerToken;
        clientOptions.AccessToken = "my-access-token";
        clientOptions.AllowInsecure = true; // Don't validate server certificate
    });
    
    // OR:

    services.AddKubeClient("my-cluster", clientOptions =>
    {
        clientOptions.ApiEndPoint = new Uri("http://localhost:8001");
        clientOptions.AuthStrategy = KubeAuthStrategy.BearerToken;
        clientOptions.AccessToken = "my-access-token";
        clientOptions.AllowInsecure = true; // Don't validate server certificate
    });
}

// To use named instances of KubeApiClient, inject INamedKubeClients.

class MyClass
{
    public MyClass(INamedKubeClients namedKubeClients)
    {
        KubeClient1 = namedKubeClients.Get("my-cluster");
        KubeClient2 = namedKubeClients.Get("another-cluster");
    }

    IKubeApiClient KubeClient1 { get; }
    IKubeApiClient KubeClient2 { get; }
}

Design philosophy

Use of code generation is limited; generated clients tend to wind up being non-idiomatic and, for a Swagger spec as large as that of Kubernetes, wind up placing too many methods directly on the client class.

KubeClient's approach is to generate model classes (see src/swagger for the Python script that does this) and hand-code the actual operation methods to provide an improved consumer experience (i.e. useful and consistent exception types).

KubeResultV1

Some operations in the Kubernetes API can return a different response depending on the arguments passed in. For example, a request to delete a v1/Pod returns the existing v1/Pod (as a PodV1 model) if the caller specifies DeletePropagationPolicy.Foreground but returns a v1/Status (as a StatusV1 model) if any other type of DeletePropagationPolicy is specified.

To handle this type of polymorphic response KubeClient uses the KubeResultV1 model (and its derived implementations, KubeResourceResultV1<TResource> and KubeResourceListResultV1<TResource>).

KubeResourceResultV1<TResource> can be implicitly cast to a TResource or a StatusV1, so consuming code can continue to use the client as if it expects an operation to return only a resource or expects it to return only a StatusV1:

PodV1 existingPod = await client.PodsV1().Delete("mypod", propagationPolicy: DeletePropagationPolicy.Foreground);
// OR:
StatusV1 deleteStatus = await client.PodsV1().Delete("mypod", propagationPolicy: DeletePropagationPolicy.Background);

If an attempt is made to cast a KubeResourceResultV1<TResource> that contains a non-success StatusV1 to a TResource, a KubeApiException is thrown, based on the information in the StatusV1:

PodV1 existingPod;

try
{
    existingPod = await client.PodsV1().Delete("mypod", propagationPolicy: DeletePropagationPolicy.Foreground);
}
catch (KubeApiException kubeApiError)
{
    Log.Error(kubeApiError, "Failed to delete Pod: {ErrorMessage}", kubeApiError.Status.Message);
}

For more information about the behaviour of KubeResultV1 and its derived implementations, see KubeResultTests.cs.

Extensibility

KubeClient is designed to be easily extensible. The KubeApiClient provides the top-level entry point for the Kubernetes API and extension methods are used to expose more specific resource clients.

Simplified version of PodClientV1.cs:

public class PodClientV1 : KubeResourceClient
{
    public PodClientV1(KubeApiClient client) : base(client)
    {
    }

    public async Task<List<PodV1>> List(string labelSelector = null, string kubeNamespace = null, CancellationToken cancellationToken = default)
    {
        PodListV1 matchingPods =
            await Http.GetAsync(
                Requests.Collection.WithTemplateParameters(new
                {
                    Namespace = kubeNamespace ?? KubeClient.DefaultNamespace,
                    LabelSelector = labelSelector
                }),
                cancellationToken: cancellationToken
            )
            .ReadContentAsObjectV1Async<PodListV1>();

        return matchingPods.Items;
    }

    public static class Requests
    {
        public static readonly HttpRequest Collection = KubeRequest.Create("api/v1/namespaces/{Namespace}/pods?labelSelector={LabelSelector?}&watch={Watch?}");
    }
}

Simplified version of ClientFactoryExtensions.cs:

public static PodClientV1 PodsV1(this KubeApiClient kubeClient)
{
    return kubeClient.ResourceClient(
        client => new PodClientV1(client)
    );
}

This enables the following usage of KubeApiClient:

KubeApiClient client;
PodListV1 pods = await client.PodsV1().List(kubeNamespace: "kube-system");

Through the use of extension methods, resource clients (or additional operations) can be declared in any assembly and used as if they are part of the KubeApiClient. For example, the KubeClient.Extensions.WebSockets package adds an ExecAndConnect method to PodClientV1.

Simplified version of ResourceClientWebSocketExtensions.cs:

public static async Task<K8sMultiplexer> ExecAndConnect(this IPodClientV1 podClient, string podName, string command, bool stdin = false, bool stdout = true, bool stderr = false, bool tty = false, string container = null, string kubeNamespace = null, CancellationToken cancellation = default)
{
    byte[] outputStreamIndexes = stdin ? new byte[1] { 0 } : new byte[0];
    byte[] inputStreamIndexes;
    if (stdout && stderr)
        inputStreamIndexes = new byte[2] { 1, 2 };
    else if (stdout)
        inputStreamIndexes = new byte[1] { 1 };
    else if (stderr)
        inputStreamIndexes = new byte[1] { 2 };
    else if (!stdin)
        throw new InvalidOperationException("Must specify at least one of STDIN, STDOUT, or STDERR.");
    else
        inputStreamIndexes = new byte[0];
    
    return await podClient.KubeClient
        .ConnectWebSocket("api/v1/namespaces/{KubeNamespace}/pods/{PodName}/exec?stdin={StdIn?}&stdout={StdOut?}&stderr={StdErr?}&tty={TTY?}&command={Command}&container={Container?}", new
        {
            PodName = podName,
            Command = command,
            StdIn = stdin,
            StdOut = stdout,
            StdErr = stderr,
            TTY = tty,
            Container = container,
            KubeNamespace = kubeNamespace ?? podClient.KubeClient.DefaultNamespace
        }, cancellation)
        .Multiplexed(inputStreamIndexes, outputStreamIndexes,
            loggerFactory: podClient.KubeClient.LoggerFactory()
        );
}

Example usage of ExecAndConnect:

KubeApiClient client;
K8sMultiplexer connection = await client.PodsV1().ExecAndConnect(
    podName: "my-pod",
    command: "/bin/bash",
    stdin: true,
    stdout: true,
    tty: true
);
using (connection)
using (StreamWriter stdin = new StreamWriter(connection.GetOutputStream(0), Encoding.UTF8))
using (StreamReader stdout = new StreamReader(connection.GetInputStream(1), Encoding.UTF8))
{
    await stdin.WriteLineAsync("ls -l /");
    await stdin.WriteLineAsync("exit");

    // Read from STDOUT until process terminates.
    string line;
    while ((line = await stdout.ReadLineAsync()) != null)
    {
        Console.WriteLine(line);
    }
}

For information about HttpRequest, UriTemplate, and other features used to implement the client take a look at the HTTPlease documentation.

Working out what APIs to call

If you want to replicate the behaviour of a kubectl command you can pass the flag --v=10 to kubectl and it will dump out (for each request that it makes) the request URI, request body, and response body.

Building

You will need to use v2.1.300 (or newer) of the .NET Core SDK to build KubeClient.

Questions / feedback

Feel free to get in touch if you have questions, feedback, or would like to contribute.

dotnet-kube-client's People

Contributors

bastianeicher avatar dependabot[bot] avatar felixfbecker avatar jonstelly avatar marcelroozekrans avatar mikecowgill avatar tintoy avatar tomsseisums avatar towmeykaw avatar typusomega avatar zeromberto avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dotnet-kube-client's Issues

Coding Rollout Pause/Resume/Rollback

It's unclear to me how I would go about acting on a deployment rollout. I'm trying to figure out how to Pause, Resume and Rollback a rollout.

Initially I thought I could Pause/Resume by patching the deployment.Spec.Paused field as seen below. But this does not work.

var depClient = GetClient().DeploymentsV1();
var dep = await depClient.Get(deploymentName);
await depClient.Update(dep.Metadata.Name, patch =>
{
   patch.Replace(d => d.Spec.Paused, true);
});

I also have no idea how I would Rollback. Any ideas on this area of the api?

Support for .NET v4.7.2

We are trying to migrate a solution to .NET 4.7.2 and are hitting issues with this component. Is there any plan to support 4.7.2 coming?

Add logging facility

Create a DelegatingHandler that logs requests and responses (only added to the pipeline when an ILogger is supplied).

To implement this, we'll need to add support for specifying one or more DelegatingHandlers when creating a KubeApiClient.

Consider adding to KubeClientOptions?

public class KubeClientOptions
{
    /// <summary>
    ///     Factories for message handlers (if any) to be added to the HTTP request pipeline.
    /// </summary>
    public List<Func<DelegatingHandler>> MessageHandlers { get; } = new List<Func<DelegatingHandler>();
}

We can then implement an AddLogging extension method for KubeClientOptions which adds the required handler.

Deserialisation from deployment yaml

I noticed when using this project to read in a static deployment yaml file, that a there is a missing yaml member alias attribute on src/KubeClient/Models/KubeResourceV1.cs

[YamlMember(Alias = "metadata")]

Without that it expects metadata to be Metadata

Helper function to map Kubernetes object kind value and apiVersion to Type

When parsing YAML files, I don't know beforehand what class I need to serialize it too. I first have to look at the kind and apiVersion properties to find out - but I don't see an easy way to get the Type (e.g. typeof(KubeDeploymentV1Beta)) from a given string kind. I could dynamically assemble the fully qualified name but it seems messy and I'm not sure about the assembly name. I think the best option would be to add a helper function, it could use Type.GetType(string) to load from the current assembly or use a big switch that can be generated (don't know what is better).

Accessing cluster Events (kubectl get events)

I'm trying to find the way to access cluster events through the api and am unable to find the right starting point. The kubectl command I'm trying to replicate is this:

kubectl get event --field-selector involvedObject.name=mypodname-69bb65fc5c-slt69

I'm sure I'm missing something obvious, if you could lead me in the right direction...

Accessing pod logs tail

I'm trying to get the last X lines of a pods logs, something like what the k8s dashboard does when viewing logs. I've experimented with the two obvious ways of acquiring the logs:

IPodClientV1.Logs()
IPodClientV1.StreamLogs()

And see they both support a limitBytes option, but both seem to start with the beginning of the logs and not the end. Some of my pods have very large logs so even if I specify a reasonable limitBytes of say 100KB I only get the very beginning of my logs.

The k8s dashboard seems to get the logs very quickly even when I have a pod with many MB of logs, so it seems like it has an intelligent way to get the end of the log stream. Any ideas how I can replicate this with this api?

Throw custom exceptions on errors

Currently, when the server returns a non-200 status code, we just propagate the low-level HTTPlease HttpException. This is a bad experience, because it does not have a very useful message:

Update-KubeResource : The request failed with unexpected status code 'InternalServerError'.
At /Users/felix/src/github.com/felixfbecker/PSKubectl/Tests/PSKubectl.Tests.ps1:175 char:37
+ ...      try { $result = $modified | Update-KubeResource -Verbose } catch ...
+                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Update-KubeResource], HttpRequestException`1
+ FullyQualifiedErrorId : HTTPlease.HttpRequestException`1[[KubeClient.Models.StatusV1, KubeClient, Version=2.2.0.0, Culture=neutral, PublicKeyToken=null]],Kubectl.Cmdlets.UpdateKubeResourceCmdlet

Kubernetes returns structured JSON information with errors (StatusV1, that is available on exception.Response). We should catch errors and rethrow a special KubeAPIException that makes use of the know strongly-typed metadata, i.e. use Status.Reason and Status.Message to construct the exception message (in the example above, that would have been Testing value /spec/template/spec/containers/0/env/1/name failed, which is a lot more specific). The rest of the status that's useful (Causes, Details etc) can then be available on a Status property on the exception. InnerException would be the raw HTTP exception.

Design proposals for TextChannelClient

Pick one or more:

  • You can call methods on the client to write text.
  • Scans for a configurable prompt string, and raises an event when it's encountered.
  • When the consumer calls channelClient.WaitForText(ChannelTypes.STDOUT, "#", text => { execClient.Send("ls -la"); }), the TextChannelClient scans the output for # and invokes the callback with information about the text (e.g. the text as well as the entire line on which it occurred).
  • Can provide extension methods for common scenarios (e.g. ExpectShellPrompt, SendCommand, etc).

JSON deserialise exception when using PodV1Client.Delete()

DELETE on a pod seems to return the deleted pod:

VERBOSE: KubeClient.KubeApiClient.Http: Completed DELETE request to 'https://localhost:6443/api/v1/namespaces/pskubectltest/pods/hello-world-bdf85c5f9-jhpxr' (OK).
VERBOSE: KubeClient.KubeApiClient.Http: Receive response body for DELETE request to 'https://localhost:6443/api/v1/namespaces/pskubectltest/pods/hello-world-bdf85c5f9-qhr89' (OK):
{"kind":"Pod","apiVersion":"v1","metadata":{"name":"hello-world-bdf85c5f9-qhr89","generateName":"hello-world-bdf85c5f9-","namespace":"pskubectltest","selfLink":"/api/v1/namespaces/pskubectltest/pods/hello-world-bdf85c5f9-qhr89","uid":"52ca2432-017b-11e9-9449-025000000001","resourceVersion":"3166","creationTimestamp":"2018-12-16T21:41:19Z","deletionTimestamp":"2018-12-16T21:44:42Z","deletionGracePeriodSeconds":30,"labels":{"app":"hello-world","pod-template-hash":"689417195"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"hello-world-bdf85c5f9","uid":"d94a88f4-017a-11e9-9449-025000000001","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-6nk4d","secret":{"secretName":"default-token-6nk4d","defaultMode":420}}],"containers":[{"name":"hello-world","image":"strm/helloworld-http:latest","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-6nk4d","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"docker-for-desktop","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-12-16T21:41:19Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-12-16T21:41:22Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-12-16T21:41:19Z"}],"hostIP":"192.168.65.3","podIP":"10.1.0.47","startTime":"2018-12-16T21:41:19Z","containerStatuses":[{"name":"hello-world","state":{"running":{"startedAt":"2018-12-16T21:41:22Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"strm/helloworld-http:latest","imageID":"docker-pullable://strm/helloworld-http@sha256:bd44b0ca80c26b5eba984bf498a9c3bab0eb1c59d30d8df3cb2c073937ee4e45","containerID":"docker://7b55dace41825a4780fd66b34e4fd0cbccc39575273df8b4392f797bb0ee6a6b"}],"qosClass":"BestEffort"}}

but PodV1Client.delete() tries to deserialise it into a StatusV1 object and therefor always throws

Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: {. Path 'status', line 1, position 1640.
   at Newtonsoft.Json.JsonTextReader.ReadStringValue(ReadType readType)
   at Newtonsoft.Json.JsonTextReader.ReadAsString()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(TextReader reader, Type objectType)
   at HTTPlease.Formatters.Json.JsonFormatter.ReadAsync(InputFormatterContext context, Stream stream)
   at HTTPlease.Formatters.ContentExtensions.ReadAsAsync[TBody](HttpContent content, IInputFormatter formatter, InputFormatterContext formatterContext)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage, IInputFormatter formatter, InputFormatterContext formatterContext)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage, IFormatterCollection formatters)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody,TError](Task`1 response, HttpStatusCode[] successStatusCodes)
   at KubeClient.ResourceClients.HttpExtensions.ReadContentAsObjectV1Async[TObject](Task`1 response, String operationDescription, HttpStatusCode[] successStatusCodes)
   at KubeClient.ResourceClients.PodClientV1.Delete(String name, String kubeNamespace, CancellationToken cancellationToken)
   at Kubectl.Cmdlets.RemoveKubePodCmdlet.deletePod(String name, String kubeNamespace, CancellationToken cancellationToken) in /Users/felix/git/PSKubectl/src/Cmdlets/RemoveKubePodCmdlet.cs:line 62

Tested by trying to delete a pod in Docker Kubernetes.

StatefulSet ResourceClient

I'm really liking the structure of this project over the "official" k8s dotnet client. Today I decided to port the a project from the other client to this but ran into a small issue. I doesn't seem there is a StatefulSet ResourceClient. From what I understand, you've hand coded those rather than rely on swagger (which is great).

I can create the StatefulSet resource client and submit a PR if it's just something you haven't got around to. Or am I missing something? Thanks!

Handling of default values

I got diffing objects working well so far, but there is one problem: Several properties in Kubernetes objects are scalars with default values. For example:

  • int ProbeV1.FailureThreshold defaults to 3
  • string ServicePortV1.Protocol defaults to "TCP"
    and many more.
    These properties are often not declared in YAML configuration.
    But when deserializing the YAML, the properties get the C# type default value, instead of the server default value. I.e. FailureThreshold is always 0, Protocol is always null.
    When diffing against the server, this will then generate a patch that sets these values to the C# default value (0/null).

How should we handle this?

  • When diffing, check if the new value is the C# default value, and exclude it from the patch. This would make it impossible to actually set a field to 0 or null. Does the API have fields anywhere that can be set to the C#/Go default value? E.g. for FailureThreshold the minimum is 1, and in other places where 0 is valid I saw them use nullable/pointer types, e.g. ConfigMapVolumeSource.DefaultMode. But there are a lot of booleans that are not nullable, and this would make it impossible to switch a boolean back to false after it was set to true on the server.
  • Apply the server defaults client side. This would break if they ever change the default. The defaults are only exposed through documentation, not in the schema.
  • Make everything nullable. This would be a bad experience when querying from the server as these are always defined.
  • Use Dictionaries or dynamic instead of strongly typed objects for deserialisation. I would like to avoid this because then we lose the formatting and autocompletion in PowerShell, and it becomes way harder to look up the merge strategies to use for properties...

Can't create a service

The TargetPort of a service is set as a string https://github.com/tintoy/dotnet-kube-client/blob/master/src/KubeClient/Models/generated/ServicePortV1.cs#L46, which fails when trying to create a service.

Exception message : "Unable to create v1/Service resource in namespace '...'.\nInvalid: Service "my-service" is invalid: spec.ports[0].targetPort: Invalid value: "9376": must contain at least one letter or number (a-z, 0-9)"

I think it needs different serialisation depending if it's an integer or string value: e.g. https://github.com/kubernetes-client/csharp/blob/master/src/KubernetesClient/generated/Models/V1ServicePort.cs#L113 and https://github.com/kubernetes-client/csharp/blob/master/src/KubernetesClient/IntstrIntOrString.cs#L6-L19

Add YAML attributes to API model classes (like JSON attributes)

I would like to be able to serialize the model classes, e.g. a Deployment, into YAML to save to files. It would be very convenient if the model classes had YamlMember attributes for YamlDotNet just like they have JsonProperty attributes, so that the properties get serialized into camelCase instead of PascalCase.

The official Kubernetes Client supports this: https://github.com/kubernetes-client/csharp/blob/HEAD/tests/KubernetesClient.Tests/YamlTests.cs#L20

The request failed with unexpected status code 'Forbidden' when connecting to AKS (dev spaces)

hi,

  • I'm running VS 2017 15.9.3 Preview 1.0.
  • I'm have installed the VS Kubernetes Tools.
  • I've deployed an ASPnet core 2.2 mvc app.

I've seen this issue: #20

However, my issue is slightly different. I create an AKS, and get Azure dev spaces
installed. I then deploy a a mvc web project to it using VS 2017.

This is the code I'm using to read my secrets with kube-client.

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseApplicationInsights()
                .ConfigureAppConfiguration(
                    configuration => GetConfigurationBuilder(configuration)
                )
                .UseStartup<Startup>()
                .UseSetting(WebHostDefaults.ApplicationKey, typeof(Program).GetTypeInfo().Assembly.FullName); // beware of this

        private static IConfigurationBuilder GetConfigurationBuilder(IConfigurationBuilder configuration)
        {
            if (Hosted.ByKubernetes)
            {
                _isConfiguredKubernetes = true;
                return configuration.AddKubeSecret(secretName: DbConnectionString,
                    clientOptions: KubeClientOptions.FromPodServiceAccount(),
                    kubeNamespace: "default",
                    reloadOnChange: true
                );
            }

            _isConfiguredKubernetes = false;

            return configuration;
        }

this code runs. however, the exception then occurs on this line:

            var host = CreateWebHostBuilder(args).Build();

Here's the exception log:

Exception thrown: 'HTTPlease.HttpRequestException`1' in System.Private.CoreLib.dll: 'The request failed with unexpected status code 'Forbidden'.'
Stack trace:
 >   at HTTPlease.FormatterResponseExtensions.<ReadContentAsAsync>d__15`2.MoveNext()
 >   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
 >   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
 >   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
 >   at KubeClient.ResourceClients.KubeResourceClient.<GetSingleResource>d__18`1.MoveNext()
 >   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
 >   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
 >   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
 >   at KubeClient.ResourceClients.SecretClientV1.<Get>d__1.MoveNext()
 >   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
 >   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
 >   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
 >   at KubeClient.Extensions.Configuration.SecretConfigurationProvider.Load()
 >   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
 >   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
 >   at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
 >   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
 >   at Application.Web.Program.Main(String[] args) in /src/application.web/Program.cs:line 25

I have previously created a secret on the AKS with the following command:

kubectl create secret generic application-web-appsettings "--from-literal=DBConnectionString='sql connection string'" -o json

I have verified that the secret exists

Delete Namespace response json deserialization exception

I am trying to delete a namespace using

StatusV1 delStatus = await client.NamespacesV1().Delete(nsName);

The api call actually works and k8s does what it needs to do. It is the deserialization of the response that fails. Some specifically the Status property.

Here is the JSON response from k8s:

{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"xxxxxxxxxx","selfLink":"/api/v1/namespaces/xxxxxxxxxx","uid":"0499e242-1a87-11e9-9f35-000d3a44ebb6","resourceVersion":"1130958","creationTimestamp":"2019-01-17T18:38:01Z","deletionTimestamp":"2019-01-17T18:40:39Z","labels":{"name":"xxxxxxxxxx"},"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{"apiVersion":"v1","kind":"Namespace","metadata":{"annotations":{},"labels":{"name":"xxxxxxxxxx"},"name":"xxxxxxxxxx"}}\n"}},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}}

Notice at the very end "status":{"phase":"Terminating"}}

The current model StatusV1 has type string in the property Status: StatusV1.cs#L47

Thanks.

New release

Hi @tintoy,

there are quite a few fixes in the trunk by now, #35, #39 and #44, that I'd love deploy.
Could you perhaps publish a new release of the lib on NuGet?

Regards
Bastian

Implement port-forward for pods

This can still use the same mechanism as exec-in-pod (K8sMultiplexer, but there are 2 streams for each forwarded port: data and error). Each channel will first receive its target port number as a 16-bit (little endian) unsigned integer.

Double-check the logic, but it looks to me like the process for forwarding ports is to listen locally for incoming TCP connections and, for each incoming connection, open a new WebSocket connection to forward the traffic for that connection (as opposed to using a single WebSocket connection and explicitly adding channels to that existing connection for each incoming TCP / UDP connection accepted by the local listener).

See the Kubernetes server-side implementation for details.

Add paging support

List operations for resources should support skip / take parameters (or a Page object that does the same thing).

Does Travis need Mono installed?

I see that .travis.yml sets mono: latest and CI spends a long time installing Mono. Is that actually needed if compilation is done with the .NET Core CLI? Can it be set to mono: none?

How to manage Ingress ?

I can find IngressSpecV1Beta1, but not an Ingress class that inherits from KubeResourceV1 or anything off KubeApiClient to create and manage ingress.

Is it not supported, or am I missing the magic?

Newtonsoft.Json.JsonSerializationException when listing pods

I am trying to connect to a Docker-for-mac Kubernetes server behind kubectl proxy.

When listing pods, I get this error:

Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'KubeClient.Models.PodListV1' because the type requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List<T>) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
Path 'kind', line 1, position 8.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(TextReader reader, Type objectType)
   at HTTPlease.Formatters.Json.JsonFormatter.ReadAsync(InputFormatterContext context, Stream stream)
   at HTTPlease.Formatters.ContentExtensions.ReadAsAsync[TBody](HttpContent content, IInputFormatter formatter, InputFormatterContext formatterContext)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage, IInputFormatter formatter, InputFormatterContext formatterContext)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage, IFormatterCollection formatters)
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody](HttpResponseMessage responseMessage)
   at KubeClient.ResourceClients.KubeResourceClient.GetResourceList[TResourceList](HttpRequest request, CancellationToken cancellationToken)
   at KubeClient.ResourceClients.PodClientV1.List(String labelSelector, String kubeNamespace, CancellationToken cancellationToken)
   at Kubectl.GetKubePodCmdlet.ProcessRecord() in /Users/felix/src/github.com/felixfbecker/PSKubectl/src/GetKubePodCmdlet.cs:line 25
   at System.Management.Automation.Cmdlet.DoProcessRecord()
   at System.Management.Automation.CommandProcessor.ProcessRecord()

Logs:

2018-08-16T19:28:42.7170010+02:00  [DBG] Performing "GET" request to 'http://127.0.0.1:8001/api/v1/namespaces/default/pods'. (cd91936f)
2018-08-16T19:28:42.7239570+02:00  [DBG] Receive response body for "GET" request to 'http://127.0.0.1:8001/api/v1/namespaces/default/pods' (OK):
"{\"kind\":\"PodList\",\"apiVersion\":\"v1\",\"metadata\":{\"selfLink\":\"/api/v1/namespaces/default/pods\",\"resourceVersion\":\"699\"},\"items\":[]}
" (29add6de)
2018-08-16T19:28:42.7240110+02:00  [DBG] Completed "GET" request to 'http://127.0.0.1:8001/api/v1/namespaces/default/pods' (OK). (be205803)

When manually requesting http://127.0.0.1:8001/api/v1/namespaces/default/pods with Postman, I get this response:

{
    "kind": "PodList",
    "apiVersion": "v1",
    "metadata": {
        "selfLink": "/api/v1/namespaces/default/pods",
        "resourceVersion": "520"
    },
    "items": []
}

Model types are missing API Group in their API Versions

Many of the auto-generated model types are missing the appropriate API Group in their API Versions.

For example DeploymentV1 is annotated with:

[KubeObject("Deployment", "v1")]

but this actually should be:

[KubeObject("Deployment", "apps/v1")]

This results in objects that are newed up and then sent to the API being rejected because they contain the wrong value in the apiVersion field.

Generic/dynamic resource client

For my implementation of kubectl apply, I get any kind of KubeResourceV1 as input. I then have to retrieve the current state from the server, compute a three-way patch (using reflection) and apply it.

Unfortunately, since the resource clients seem to only be accessible through extension methods, and KubeResourceClient only has protected methods. Therefor I have to write a big if/else and handle every possible resource type:

if (Resource is DeploymentV1Beta1) {
    DeploymentV1Beta1 modified = Resource as DeploymentV1Beta1;
    DeploymentV1Beta1 current = await client.DeploymentsV1Beta1().Get(Resource.Metadata.Name, Resource.Metadata.Namespace, cancellationToken);
    DeploymentV1Beta1 original;
    string originalJson = current.Metadata.Annotations[lastAppliedConfigAnnotation];
    if (!String.IsNullOrEmpty(originalJson)) {
        original = JsonConvert.DeserializeObject<DeploymentV1Beta1>(originalJson);
    } else {
        original = modified;
    }

    Action<JsonPatchDocument<DeploymentV1Beta1>> patchAction = deploymentPatch => {
        var patch = new JsonPatchDocument();
        diff(current, modified, patch, ignoreDeletions: true);
        diff(original, modified, patch, ignoreAdditionsAndModifications: true);
        foreach (var operation in patch.Operations) {
            deploymentPatch.Operations.Add(new Operation<DeploymentV1Beta1>(operation.op, operation.path, operation.from, operation.value));
        }
    };

    if (ShouldProcess(Resource.Metadata.Name, "patch")) {
        await client.DeploymentsV1Beta1().Update(Resource.Metadata.Name, patchAction, Resource.Metadata.Namespace, cancellationToken);
    }
} else if (Resource is ServiceV1) {
    ServiceV1 modified = Resource as ServiceV1;
    ServiceV1 current = await client.ServicesV1().Get(Resource.Metadata.Name, Resource.Metadata.Namespace, cancellationToken);
    ServiceV1 original;
    string originalJson = current.Metadata.Annotations[lastAppliedConfigAnnotation];
    if (!String.IsNullOrEmpty(originalJson)) {
        original = JsonConvert.DeserializeObject<ServiceV1>(originalJson);
    } else {
        original = modified;
    }

    Action<JsonPatchDocument<ServiceV1>> patchAction = deploymentPatch => {
        var patch = new JsonPatchDocument();
        diff(current, modified, patch, ignoreDeletions: true);
        diff(original, modified, patch, ignoreAdditionsAndModifications: true);
        foreach (var operation in patch.Operations) {
            deploymentPatch.Operations.Add(new Operation<ServiceV1>(operation.op, operation.path, operation.from, operation.value));
        }
    };

    if (ShouldProcess(Resource.Metadata.Name, "patch")) {
        await client.ServicesV1().Update(Resource.Metadata.Name, patchAction, Resource.Metadata.Namespace, cancellationToken);
    }
    await client.ServicesV1().Update(Resource.Metadata.Name, patchAction, Resource.Metadata.Namespace, cancellationToken);
} else if (Resource is PersistentVolumeClaimV1) {
    // ... repeat for every single resource type ...

It would be awesome if instead, I could use a dynamic KubeResouceClient like this:

IKubeResourceClient resourceClient = client.ResourceClient(Resource.GetType());
if (!(resourceClient is IGetClient)) {
    throw new Exception($"Resource kind {Resource.Kind} is not gettable");
}
if (!(resourceClient is IPatchClient)) {
    throw new Exception($"Resource kind {Resource.Kind} is not patchable");
}
KubeResourceV1 current = await (resourceClient as IGetClient).Get(Resource.Metadata.Name, Resource.Metadata.Namespace, cancellationToken);
KubeResourceV1 original;
string originalJson = current.Metadata.Annotations[lastAppliedConfigAnnotation];
if (!String.IsNullOrEmpty(originalJson)) {
    original = JsonConvert.DeserializeObject(originalJson, Resource.GetType()) as KubeResourceV1;
} else {
    original = Resource;
}

Action<JsonPatchDocument> patchAction = patch => {
    diff(current, Resource, patch, ignoreDeletions: true);
    diff(original, Resource, patch, ignoreAdditionsAndModifications: true);
};

if (ShouldProcess(Resource.Metadata.Name, "patch")) {
    await (resourceClient as IPatchClient).Update(Resource.Metadata.Name, patchAction, Resource.Metadata.Namespace, cancellationToken);
}

I am not sure if this actually works or if this is idiomatic API design in C# - wdyt?

Add abstraction for combining multiplexed read / write streams

For example:

K8sMultiplexer multiplexer;

using (Stream stdin = multiplexer.GetInputStream(K8sChannel.StdIn))
using (Stream stdout = multiplexer.GetOutputStream(K8sChannel.StdOut))
using (CompositeStream stream = new CompositeStream(readFrom: stdout, writeTo: stdin))
{
    // Use stream.
}

Token Authentication Fails, Token in Config isn't base64 encoded

I've got a microk8s cluster and by default the admin user is set up for basic username/password authentication. Since this client doesn't support username/password authentication I generated a token for authentication and changed my user entry to use the token.

I got the token by running the following PowerShell script:

$secrets = kubectl -n kube-system get secret -o=json | ConvertFrom-Json;
$admin = $secrets.items.Where( { $_.metadata.name -like 'admin-user*' });
$token = [Text.Encoding]::ASCII.GetString([Convert]::FromBase64String($admin.data.token));

If I take the value from that script, I can use the token to log into the dashboard, and placing the token in the kubernetes config file, kubectl get all works as expected but the token doesn't seem to be base64 encoded (It has non-base64 characters). If I try to use this kubernetes client, I get this exception:

System.FormatException : The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
   at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
   at System.Convert.FromBase64String(String s)
   at KubeClient.Extensions.KubeConfig.Models.UserIdentityConfig.GetRawToken() in C:\code\gh\dotnet-kube-client\src\KubeClient.Extensions.KubeConfig\Models\UserIdentityConfig.cs:line 64
   at KubeClient.K8sConfig.ConfigureKubeClientOptions(KubeClientOptions kubeClientOptions, String kubeContextName, String defaultKubeNamespace) in C:\code\gh\dotnet-kube-client\src\KubeClient.Extensions.KubeConfig\K8sConfig.cs:line 200
   at KubeClient.K8sConfig.ToKubeClientOptions(String kubeContextName, String defaultKubeNamespace, ILoggerFactory loggerFactory) in C:\code\gh\dotnet-kube-client\src\KubeClient.Extensions.KubeConfig\K8sConfig.cs:line 150
   at KubeClient.Extensions.KubeConfig.Tests.K8sConfigLocationTests.ConnectToHome() in C:\code\gh\dotnet-kube-client\test\KubeClient.Extensions.KubeConfig.Tests\K8sConfigLocationTests.cs:line 81
--- End of stack trace from previous location where exception was thrown ---

So I tried to base64 encode the value. More Powershell:

[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes('TOKEN_GOES_HERE')

And replaced the token value in my config file with the encoded value. When I do that, kubectl get all fails to authenticate and so does this client.

My assumption is that only http bearer strings are base64 encoded. If so, a change like the following seems like it would make sense. Does this look right? My assumption could obviously be wrong and this might break something. I tried looking through the kubectl code, but the phrases bearer and token show up a ton in the code so I wasn't able to find what I was looking for.

image

Working with custom objects from CRDs

I wondered if you could guide me to an example .. I am now starting to create my own resources (CRDs) in K8S ..
For example - /apis/things.example.com/v1/namespaces/default/things

apiVersion: "things.example.com"
kind: MyThing
metadata:
    name: my-first-thing
spec:
    thingState: draft
    thingSizes:
        - xl
        - xxl

I was wondering what the best way to handle CRDs in dotnet-kube-client and to have it retrieve (and if possible) deserialise to a C# object that represents MyThing.

I like the way you have put the library together and I would like to move to this, so I am just evaluating how to do certain things (I appreciate this is a relatively new feature of K8S)

In the official K8S CSharp client the closest I can get is GetNamespacedCustomObject that just returns me an object .. which isnt terribly helpful :)

Persistent Volumes aren't namespace-scoped, Client calls fail with 404

Trying to list persistent volumes gets me a 404:

KubeClient.KubeApiException : Unable to list PersistentVolume (v1) resources (HTTP status NotFound).
NotFound: the server could not find the requested resource
---- HTTPlease.HttpRequestException`1[[KubeClient.Models.StatusV1, KubeClient, Version=2.2.8.0, Culture=neutral, PublicKeyToken=null]] : The request failed with unexpected status code 'NotFound'.
   at KubeClient.ResourceClients.KubeResourceClient.GetResourceList[TResourceList](HttpRequest request, CancellationToken cancellationToken)
   at KubeClient.ResourceClients.PersistentVolumeClientV1.List(String labelSelector, String kubeNamespace, CancellationToken cancellationToken)

I think the change starts with the below diff, but I'm not 100% sure. If this looks right then I guess we'd also need to remove the kubeNamespace parameter from the methods on PersistenVolumeClientV1? If I load http://localhost:8001/api/v1/ in a browser I can see a namespaced: true/false property and it's false for PersistentVolumes, so this seems right. Is there any more complexity to this change other than changing PersistentVolumeClient? If not I can submit a PR this evening or tomorrow.

...\gh\dotnet-kube-client [feature/pv-fix +0 ~1 -0 !]> git diff
diff --git a/src/KubeClient/ResourceClients/PersistentVolumeClientV1.cs b/src/KubeClient/ResourceClients/PersistentVolumeClientV1.cs
index 15c78c2..b4164db 100644
--- a/src/KubeClient/ResourceClients/PersistentVolumeClientV1.cs
+++ b/src/KubeClient/ResourceClients/PersistentVolumeClientV1.cs
@@ -173,12 +173,12 @@ namespace KubeClient.ResourceClients
             /// <summary>
             ///     A collection-level PersistentVolume (v1) request.
             /// </summary>
-            public static readonly HttpRequest Collection   = KubeRequest.Create("api/v1/namespaces/{Namespace}/persistentvolumes?labelSelector={LabelSelector?}&watch={Watch?}");
+            public static readonly HttpRequest Collection   = KubeRequest.Create("api/v1/persistentvolumes?labelSelector={LabelSelector?}&watch={Watch?}");

             /// <summary>
             ///     A get-by-name PersistentVolume (v1) request.
             /// </summary>
-            public static readonly HttpRequest ByName       = KubeRequest.Create("api/v1/namespaces/{Namespace}/persistentvolumes/{Name}");
+            public static readonly HttpRequest ByName       = KubeRequest.Create("api/v1/persistentvolumes/{Name}");
         }
     }

Expose Kubernetes OpenAPI extensions as attributes on members

To properly patch resources (like a kubectl apply), it is necessary to know how the fields (especially lists) in the resources are intended to be merged. This is exposed as OpenAPI extensions in x-kubernetes-patch-strategy and x-kubernetes-patch-merge-key.

Question: Using DeploymentV1.Update()

Are there any sample snippets on how to update a deployment? Specifically I'm struggling to figure out how to start a pod scale-out by increasing the Replicas count. The syntax of the api is tripping me up quite thoroughly...

Int32OrStringV1 throwing exception on deployment liveness probe

I'm using KubeClient to list deployments and getting an exception when processing a liveness probe.

KubeClient version: 2.2.7

Kubernetes Version info:

Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:00:57Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

The exception looks like this:

Newtonsoft.Json.JsonReaderException : Could not convert string to integer: scheme. Path 'items[0].spec.template.spec.containers[0].livenessProbe.httpGet.port', line 1, position 1875.
   at Newtonsoft.Json.JsonReader.ReadInt32String(String s)
   at Newtonsoft.Json.JsonTextReader.FinishReadQuotedNumber(ReadType readType)
   at Newtonsoft.Json.JsonTextReader.ReadAsInt32()
   at KubeClient.Models.Converters.Int32OrStringV1Converter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in ...\dotnet-kube-client\src\KubeClient\Models\Converters\Int32OrStringV1Converter.cs:line 75

From debugging it seems to be the liveness check for nginx ingress controller (deployed with Helm from here):

kubectl describe deployment/ingress-private-nginx-ingress-controller


Name:                   ingress-private-nginx-ingress-controller
Namespace:              default
CreationTimestamp:      Mon, 04 Mar 2019 21:49:01 -0600
Labels:                 app=nginx-ingress
                        chart=nginx-ingress-1.1.4
                        component=controller
                        heritage=Tiller
                        release=ingress-private
Annotations:            deployment.kubernetes.io/revision: 2
Selector:               app=nginx-ingress,component=controller,release=ingress-private
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:           app=nginx-ingress
                    component=controller
                    release=ingress-private
  Service Account:  ingress-private-nginx-ingress
  Containers:
   nginx-ingress-controller:
    Image:       quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.21.0
    Ports:       80/TCP, 443/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP
    Args:
      /nginx-ingress-controller
      --default-backend-service=default/ingress-private-nginx-ingress-default-backend
      --election-id=ingress-controller-leader
      --ingress-class=private
      --configmap=default/ingress-private-nginx-ingress-controller
      --tcp-services-configmap=default/ingress-private-nginx-ingress-tcp
      --enable-ssl-passthrough=true
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:        (v1:metadata.name)
      POD_NAMESPACE:   (v1:metadata.namespace)
    Mounts:           <none>
  Volumes:            <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   ingress-private-nginx-ingress-controller-5775b9579f (1/1 replicas created)
Events:          <none>

the liveness port for 10254 is where the exception happens. I was able to capture the JSON for this deployment by replacing KubeResourceClient.cs line 178 with var json = await responseMessage.Content.ReadAsStringAsync();

{
  "kind": "DeploymentList",
  "apiVersion": "apps/v1",
  "metadata": { "selfLink": "/apis/apps/v1/namespaces/default/deployments", "resourceVersion": "1959501" },
  "items": [
    {
      "metadata": {
        "name": "ingress-private-nginx-ingress-controller",
        "namespace": "default",
        "selfLink": "/apis/apps/v1/namespaces/default/deployments/ingress-private-nginx-ingress-controller",
        "uid": "9d22afb8-3ef9-11e9-aa7e-00155d10f745",
        "resourceVersion": "1258855",
        "generation": 2,
        "creationTimestamp": "2019-03-05T03:49:01Z",
        "labels": { "app": "nginx-ingress", "chart": "nginx-ingress-1.1.4", "component": "controller", "heritage": "Tiller", "release": "ingress-private" },
        "annotations": { "deployment.kubernetes.io/revision": "2" }
      },
      "spec": {
        "replicas": 1,
        "selector": { "matchLabels": { "app": "nginx-ingress", "component": "controller", "release": "ingress-private" } },
        "template": {
          "metadata": { "creationTimestamp": null, "labels": { "app": "nginx-ingress", "component": "controller", "release": "ingress-private" } },
          "spec": {
            "containers": [
              {
                "name": "nginx-ingress-controller",
                "image": "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.21.0",
                "args": [
                  "/nginx-ingress-controller",
                  "--default-backend-service=default/ingress-private-nginx-ingress-default-backend",
                  "--election-id=ingress-controller-leader",
                  "--ingress-class=private",
                  "--configmap=default/ingress-private-nginx-ingress-controller",
                  "--tcp-services-configmap=default/ingress-private-nginx-ingress-tcp",
                  "--enable-ssl-passthrough=true"
                ],
                "ports": [
                  { "name": "http", "containerPort": 80, "protocol": "TCP" },
                  { "name": "https", "containerPort": 443, "protocol": "TCP" }
                ],
                "env": [
                  { "name": "POD_NAME", "valueFrom": { "fieldRef": { "apiVersion": "v1", "fieldPath": "metadata.name" } } },
                  { "name": "POD_NAMESPACE", "valueFrom": { "fieldRef": { "apiVersion": "v1", "fieldPath": "metadata.namespace" } } }
                ],
                "resources": {},
                "livenessProbe": {
                  "httpGet": { "path": "/healthz", "port": 10254, "scheme": "HTTP" },
                  "initialDelaySeconds": 10,
                  "timeoutSeconds": 1,
                  "periodSeconds": 10,
                  "successThreshold": 1,
                  "failureThreshold": 3
                },
                "readinessProbe": {
                  "httpGet": { "path": "/healthz", "port": 10254, "scheme": "HTTP" },
                  "initialDelaySeconds": 10,
                  "timeoutSeconds": 1,
                  "periodSeconds": 10,
                  "successThreshold": 1,
                  "failureThreshold": 3
                },
                "terminationMessagePath": "/dev/termination-log",
                "terminationMessagePolicy": "File",
                "imagePullPolicy": "IfNotPresent",
                "securityContext": { "capabilities": { "add": ["NET_BIND_SERVICE"], "drop": ["ALL"] }, "runAsUser": 33, "procMount": "Default" }
              }
            ],
            "restartPolicy": "Always",
            "terminationGracePeriodSeconds": 60,
            "dnsPolicy": "ClusterFirst",
            "serviceAccountName": "ingress-private-nginx-ingress",
            "serviceAccount": "ingress-private-nginx-ingress",
            "securityContext": {},
            "schedulerName": "default-scheduler"
          }
        },
        "strategy": { "type": "RollingUpdate", "rollingUpdate": { "maxUnavailable": 1, "maxSurge": 1 } },
        "revisionHistoryLimit": 10,
        "progressDeadlineSeconds": 2147483647
      },
      "status": {
        "observedGeneration": 2,
        "replicas": 1,
        "updatedReplicas": 1,
        "readyReplicas": 1,
        "availableReplicas": 1,
        "conditions": [
          {
            "type": "Available",
            "status": "True",
            "lastUpdateTime": "2019-03-05T03:49:01Z",
            "lastTransitionTime": "2019-03-05T03:49:01Z",
            "reason": "MinimumReplicasAvailable",
            "message": "Deployment has minimum availability."
          }
        ]
      }
    }
  ]
}

Serialization sets invalid default values

Assume the following deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
        - name: test
          image: nginx

Creating this with kubectl apply succeeds.

Trying to do the same with KubeClient fails with an "unprocessable entity" error:

var deployment = new DeploymentV1Beta1
{
    Metadata = new ObjectMetaV1 {Name = "test"},
    Spec = new DeploymentSpecV1Beta1
    {
        Selector = new LabelSelectorV1
        {
            MatchLabels = {["app"] = "test"}
        },
        Template = new PodTemplateSpecV1
        {
            Metadata = new ObjectMetaV1
            {
                Labels = {["app"] = "test"}
            },
            Spec = new PodSpecV1
            {
                Containers =
                {
                    new ContainerV1
                    {
                        Name = "test",
                        Image = "nginx"
                    }
                }
            }
        }
    }
}));
client.Create(deployment);

Serializing the deployment object to JSON with JsonConvert.SerializeObject(deployment) and then converting it to YAML for better readability gives the following (slightly abbreviated):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  uid: 
  clusterName: 
  generateName: 
  # ...
spec:
  paused: false
  selector:
    matchExpressions: []
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
      uid: 
      clusterName: 
      generateName: 
      name: 
      # ...
    spec:
      hostIPC: false
      hostPID: false
      hostname: 
      nodeName: 
      # ...
      containers:
      - name: test
        image: nginx
        command: []
        lifecycle: 
        livenessProbe:        
        readinessProbe: 
        # ...
      hostAliases: []
      imagePullSecrets: []
      initContainers: []
      readinessGates: []
      # ...
  rollbackTo: 
  minReadySeconds: 0
  progressDeadlineSeconds: 0
  replicas: 0
  revisionHistoryLimit: 0
  strategy: 
status: 

Trying this with kubectl apply fails with: unknown field "readinessGates" in io.k8s.api.core.v1.PodSpec
After removing that field it fails with: spec.progressDeadlineSeconds: Invalid value: 0: must be greater than minReadySeconds.

I suppose the underlying problem is, that fields like DeploymentV1.ProgressDeadlineSeconds have documentation like Defaults to 600s. but these default values are not represented in code and the fields in question are not nullable.

Need help to deserialise yaml

Hi,

Sorry this is such a basic question. I've been using the other dotnet k8s library and would like to switch to yours, but I'm failing at the first hurdle.

I can't work out how to read in my .yaml files (was doing with await Yaml.LoadFromFileAsync<V1Service>(...) in KubernetesClient, which was working fine).

I'm trying things like

Deserializer deserializer = new DeserializerBuilder()
  .Build();

... deserializer.Deserialize<ServiceV1>(File.ReadAllText(...)); 

but always get exceptions like: "Property 'selector' not found on type 'KubeClient.Models.ServiceSpecV1'."

What's the magic salt I need to read in a .yaml?

reload on configmap change not working

I tried setting it up in a demo asp.net core app, the configuration does not reload when the config map changes. Do you have any working samples for ASP.NET Core Web App

The request failed with unexpected status code 'Forbidden' when connecting to GKE

First of all, thank you for this library! I totally agree with the design philosophy and the API feels so much more ergonomic than the official client.

I tried it to get all pods in a kubectl get pods fashion (using ~/.kube/config):

K8sConfig config = K8sConfig.Load();
KubeClientOptions clientOptions = config.ToKubeClientOptions(
    defaultKubeNamespace: "default"
);
clientOptions.LogHeaders = true;
clientOptions.LogPayloads = true;
var loggerFactory = new LoggerFactory();
loggerFactory.AddFile("test.log", LogLevel.Trace);
client = KubeApiClient.Create(clientOptions, loggerFactory);

but I always get this error:

HTTPlease.HttpRequestException`1[KubeClient.Models.StatusV1]: The request failed with unexpected status code 'Forbidden'.
   at HTTPlease.FormatterResponseExtensions.ReadContentAsAsync[TBody,TError](HttpResponseMessage responseMessage, HttpStatusCode[] successStatusCodes)
   at KubeClient.ResourceClients.KubeResourceClient.GetResourceList[TResourceList](HttpRequest request, CancellationToken cancellationToken)
   at KubeClient.ResourceClients.PodClientV1.List(String labelSelector, String kubeNamespace, CancellationToken cancellationToken)
   at Kubectl.GetKubePodCmdlet.ProcessRecord() in /Users/felix/src/github.com/felixfbecker/PSKubectl/src/GetKubePodCmdlet.cs:line 23
   at System.Management.Automation.Cmdlet.DoProcessRecord()
   at System.Management.Automation.CommandProcessor.ProcessRecord()

The cluster is hosted on Google Kubernetes Engine.
kubectl get pods lists the pods successfully.

These are the logs:

2018-08-16T17:07:16.9259220+02:00  [DBG] Performing "GET" request to 'https://35.202.230.255/api/v1/namespaces/default/pods'. (cd91936f)
2018-08-16T17:07:18.8543420+02:00  [DBG] Receive response body for "GET" request to 'https://35.202.230.255/api/v1/namespaces/default/pods' (Forbidden):
"{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"pods is forbidden: User \\"system:anonymous\\" cannot list pods in the namespace \\"default\\": No policy matched.\nUnknown user \\"system:anonymous\\"\",\"reason\":\"Forbidden\",\"details\":{\"kind\":\"pods\"},\"code\":403}
" (29add6de)
2018-08-16T17:07:18.8547270+02:00  [DBG] Completed "GET" request to 'https://35.202.230.255/api/v1/namespaces/default/pods' (Forbidden). (be205803)

This is the auth config in kubeconfig:

user:
    auth-provider:
      config:
        access-token: REDACTED
        cmd-args: config config-helper --format=json
        cmd-path: /Users/felix/google-cloud-sdk/bin/gcloud
        expiry: 2018-08-15T18:06:19Z
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

Any idea why kubectl get pods works but KubeClient fails?

Testability - Missing abstraction

Hi,

we really appreciate your lib and want to use it, but we are currently facing some hard times testing our code using your lib.
The lib provides no interfaces, which means that we cannot mock your implementations.

As we are developing in a CI environment, we strongly need to provide unittests (not only integrationtests).

Do you think there is a chance to introduce an appropiate level of abstraction, especially for the KubeApiClient and the ResourceClients?

Thanks in advance
Andy

Annotate non-updatable properties

I am finding myself to write quite a few hardcoded special cases to prevent non-updatable properties from being updated. Would it be possible somehow to mark these as readonly in the field so I can introspect that instead? Do Yaml.NET and Json.NET support populating readonly fields?

This is related to #28 because there are some fields that don't have a "default value" but are populated by the system and cannot be updated, and we don't want those properties to show up in the patch either (to not be reset to their zero value). Examples: ObjectMetaV1.Uid, ContainerV1.TerminationMessagePolicy

Integration tests

Bugs like #51 could be prevented by having tests that run against an actual Kubernetes cluster. This would catch wrong assumptions like return types of certain endpoints.

It's a fair amount of work to get right, but I actually already have this set up for PSKubectl. It runs minikube in Travis, kubectl apply's test resources, runs commands and then does assertions against output of kubectl -o json. Tests are written in PowerShell with Pester which makes interacting with kubectl very easy.

Alternatively we could just consider PSKubectl an external test suite of this, maybe even trigger a build on every published version from a branch. Or CI here could run the test suite of PSKubectl by cloning the GitHub repo.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.