mariotoffia / fluentdocker Goto Github PK

Encapsulate the native docker host and docker machine (and later on the new "native"/virtualized docker host) into services. This means that it will discover the local installation and via docker-machine discover any additional services. Each would yield one service instance start/stoppable and query for docker parameters.

In addition it shall be possible to create a new instance, via docker-machine, of a host service where new containers may be deployed.

It shall also be possible to create a new instance via a docker url.

Create docker client commands to use the docker binary to submit commands.

Hi,

Today I started working with .Mount to mount folders inside the container.
I'm working in Windows with linux containers (default options)

If I try to mount a directory with a space in the path I get weird errors by the docker deamon. I'm quite sure the problem is a wrong escaping of the string with the path.

Maybe could be better to give the input path as an instance of FileInfo class or to add quotes or escaping characters

When reusing a existing container it will not apply e.g. KeepContainer etc since the DockerHostService is created with the default parameters.

The correct behavior is to apply the parameters from the builder.

In order to read from logs or events both eternal streams or non eternal streams.

Hi,

I am listing the available containers with _docker.GetImages(), putting the result into a list and casting each element to DockerContainerService.

How it is possible from an object like that to have the Image name which the image was created from?

When running ComposeUp or ComposeDown, the command fails if the YAML file path/name contains spaces.

Commands.Compose.ComposeUp(dockerUri, filePathWithSpacesInName);

Commands.Compose.ComposeDown(dockerUri, composeFile: filePathWithSpacesInName);

I think it just needs double-quotes around the file parameter:

    public static CommandResponse<IList<string>> ComposeDown(this DockerUri host, [...etc.])
    {
      var args = $"{host.RenderBaseArgs(certificates)}";
      if (!string.IsNullOrEmpty(composeFile))
      {
        args += $" -f {composeFile}";  // <---------- here
      }

I think the line noted should be

args += $" -f \"{composeFile}\"";

(Sorry for no pull request, but even after getting the Postgres container running, I can't get most of the tests to work on my machine, so I wouldn't be able to know whether the changes broke anything.)

Today, FluentDocker needs to have the docker binaries somewhere in the path or explicitly pointed out. A driver model that implements docker binaries and then later on allows for HTTP traffic only drivers to be loaded and used. This will enable communication with the docker daemon without using the binaries on the filesystem as a requirement.

Still the docker binary driver should be delivered with the main assembly so only one assembly is needed to provide exiting functionality. However if other drivers, it can be loaded and composed. A investigation how wide spread MEF on different system. That could be a candidate to use as weaver.

One problem I found when I installed and tested the nuget package in the .NET Core 2.0 MSTest project. When I run the code like below, runtime throws 'MissingMethodException'.

new Builder().UseContainer()
            .UseImage("mysql-server:5.7")
...
;

System.MissingMethodException
  HResult=0x80131513
  Message=Method not found: 'System.IServiceProvider Microsoft.Extensions.DependencyInjection.ServiceCollectionContainerBuilderExtensions.BuildServiceProvider(Microsoft.Extensions.DependencyInjection.IServiceCollection)'.
  Source=Ductus.FluentDocker
  StackTrace:
   at Ductus.FluentDocker.Common.Logger.<>c.<.cctor>b__2_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at Ductus.FluentDocker.Common.Logger.Log(String message)
   at Ductus.FluentDocker.Executors.ProcessExecutor`2.Execute()
   at Ductus.FluentDocker.Services.Hosts.Discover()
   at Ductus.FluentDocker.Builders.Builder.FindOrDefineHostBuilder()
   at Ductus.FluentDocker.Builders.Builder.UseContainer()
...
   at Ductus.FluentDocker.MsTest.FluentDockerTestBase.Initialize()

The exception says that the runtime can not find the extension method ServiceCollectionContainerBuilderExtensions.BuildServiceProvider that returns an instance of type IServiceProvider.

In fact, in .NET Core 2.0, Microsoft has created a breaking change with a version of the method that returns the ServiceProvider class instead of the IServiceProvider interface, and it seems that a MissingMethodException is being thrown.

It seems to be possible to use this library in a .NET Core 2.0 project only by fixing this problem.

Have just begun learning docker, and downloaded versión 17.3.1-ce. I am trying to use FluentDocker to setup and throw machines at integration tests. Looks like a good scenario for the library

However, trying to run Hosts.Discover() explodes with an ArgumentNullException.

Downloaded your code and run MachineServiceTests.DiscoverShouldReturnNativeWhenSuchIsPresent

Call Stack is

Newtonsoft.Json.Linq.Extensions.Value(IEnumerable1 value) Ductus.FluentDocker.Executors.Parsers.MachineInspectResponseParser.Process(ProcessExecutionResult response) in MachineInspectResponseParser.cs: line: 28 Ductus.FluentDocker.Executors.ProcessExecutor2.Execute() in ProcessExecutor.cs: line: 68
Ductus.FluentDocker.Commands.Machine.Inspect(String machine) in Machine.cs: line: 23
Ductus.FluentDocker.Services.Hosts.<>c.b__0_0(MachineLsResponse machine) in Hosts.cs: line: 23
System.Linq.Enumerable.<>c__DisplayClass7_03.<CombineSelectors>b__0(TSource x) System.Linq.Enumerable.WhereSelectListIterator2.MoveNext()
System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection)
Ductus.FluentDocker.Services.Hosts.Discover() in Hosts.cs: line: 22
Ductus.FluentDockerTest.ServiceTests.MachineServiceTests.DiscoverShouldReturnNativeWhenSuchIsPresent() in MachineServiceTests.cs: line: 21

Checking it closely the problem lies at MachineInspectResponseParser.
To retrieve memory, it does

MemorySizeMb = j["Driver"]["Memory"].Value(),

but the objet it is trying to parse doesn't have a memory entry inside Driver. It has MemSize instead.

So it explodes

Hi,

It is still me! I'm sorry for so many issues but I'm using really a lot your library and I always have questions :)

I'm creating images from Dockerfile with the example you provided in previous issue and all works correctly.
new Ductus.FluentDocker.Builders.Builder() .DefineImage("ImageName").ReuseIfAlreadyExists() .FromFile("DockerfilePath") .Build();

How can I assure I'm using the right DockerUri? In this command it is not specified the DockerHostService or DockerUri.
I'm asking because I'll need to operate with more than one docker instance, on remote hosts, so I want to create specific images on different hosts.

Thank you for everything :)

This should support both fluent instructions to weave containers and networks as well as providing yaml files for docker-compose. If docker-compose is used those containers shall be reflected into a weave service instance (mesh) and be controllable from those.

Sometimes I get NullReferenceException while getting list of all containers via IHostService.GetContainers()

The problem is that Host.InspectContainer(id, Certificates).Data returns null for some reason. However Host.InspectContainer(id, Certificates).Success returns true at the same time.

Any ideas what could be wrong?
Thanks in advance

In order to require or create services a new Service (fluent API) needs to be added. It shall be possible to weave this into one or more docker container definitions.

This API shall support to handle different docker definitions under different services, thus end up in different host containers.

Related to #33, Discover fails for me, since the first returned host (a DigitalOcean host) does not contain DiskSize or CPU under its Driver object, which makes it fail in MachineInspectResponseParser:

Error Message:
 System.ArgumentNullException : Value cannot be null.
Parameter name: value
Stack Trace:
   at Newtonsoft.Json.Linq.Extensions.Value[T,U](IEnumerable`1 value)
   at Ductus.FluentDocker.Executors.Parsers.MachineInspectResponseParser.Process(ProcessExecutionResult response)
   at Ductus.FluentDocker.Executors.ProcessExecutor`2.Execute()
   at Ductus.FluentDocker.Commands.Machine.Inspect(String machine)
   at Ductus.FluentDocker.Services.Hosts.<>c.<Discover>b__0_0(MachineLsResponse machine)
   at System.Linq.Enumerable.<>c__DisplayClass151_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at Ductus.FluentDocker.Services.Hosts.Discover()
   at Ductus.FluentDocker.Builders.Builder.FindOrDefineHostBuilder()
   at Ductus.FluentDocker.Builders.Builder.UseContainer()

If it seems like a good idea to you, I could try making a PR with some more general error handling for missing properties; I don't know enough about the lib to know what the best way to handle missing DiskSize´ or CPU` is, though - should we fail (with a better error message), or should this be accepted?

Refactor to capture stdout/err and propagate exit code and the whole process to response parsers to allow them to do better error handling.

When I run my code in a container I get an error:

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Ductus.FluentDocker.Extensions.CommandExtensions' threw an exception. ---> Ductus.FluentDocker.Common.FluentDockerException: Failed to find docker client binary - please add it to your path

The error occurs on my first line of code:
var hosts = new Hosts().Discover();

Please assist.

Regards
Jolanda

In powershell I would run: docker run --name testing --restart always testapp:load1.0

How can I add the --restart always to the statement below?

        _docker.Host.Run("testapp:load1.0",
        new ContainerCreateParams()
        {
            Name = "testing113",
        }
        , _docker.Certificates);

Please assist.

Regards
Jolanda

I'm trying to run integration tests that use FluentDocker inside GitLab's CI pipeline, but I get this error:

System.TypeInitializationException : The type initializer for 'Ductus.FluentDocker.Extensions.CommandExtensions' threw an exception.
  ----> Ductus.FluentDocker.Common.FluentDockerException : Failed to find docker client binary - please add it to your path
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
   at Ductus.FluentDocker.Extensions.CommandExtensions.IsMachineBinaryPresent()
   at Ductus.FluentDocker.Services.Hosts.Discover()

Any idea how to fix this?

I presume this has something to do with how GitLab jobs are working: essentially GitLab creates docker image with our scripts in it, so test project is actually running inside docker, and the whole thing becomes docker-in-docker.

Hi,

I've started using your library yesterday. It is very useful and looks like there is an option for everything.

The only thing I cannot find is how to create an image from a Dockerfile.
Is this function available?

Thank you

Can I disable the Logger output? Sometimes it's handy, but sometimes it just clutters my Debug window.

Is there some kind of configuration setting to turn it off?

Is there a way to declare container mounts to be disposable? Container dispose seems to remove only the container itself, but the mounts that were created with that container are left orphaned. It is useful in case I want to reuse the same mount with the same data on the next run, but there are times when I'd like to clear data that has been persisted in the mount between test runs.

Add fluent API to create docker host (via machine) or define requirements on the docker hosts to be verified.

Use the docker machine binary to execute docker hosts and manage them (if not native).

Shouldn't that use Path.Combine or at least Path.DirectorySeparatorChar?
Otherwise this is very Windows-centric code.

When doing a connect onto the docker daemon (using docker for windows beta) it now resolves to a IPv6 address by default (Currently there's a fix to check for an IPv4 address and use that). By using this IPEndpoint IPv6 address and a port will make socket.Connect to succeed even if the port is not currently open.

This needs to be investigated why the Connect method succeeds and make a fix to support both IPv4 and v6.

I got a Win32Exception when using FluentDocker 2.2.3 from NuGet in a .NET Core 2.0 console application on Mac with Docker installed.

Code:

var hosts = new Hosts().Discover();

Error: System.ComponentModel.Win32Exception, "No such file or directory" on ResolvePath(filename).

Is this supposed to work on .NET Core?

Add commands to control the docker-compose binary.

Filed as an issue at docker machie GitHub repo.

Non docker toolbox docker-machine complains about it cannot contact the daemon. Thus the command responder (correctly) gather std-error log. This behavior is problematic since it is possible to to use the newly created machine and thus should not produce an error log.

The error message:
"This machine has been allocated an IP address, but Docker Machine could not reach it successfully.
SSH for the machine should still work, but connecting to exposed ports, such as the Docker daemon port (usually :2376), may not work properly.
You may need to add the route manually, or use another related workaround.
This could be due to a VPN, proxy, or host file configuration issue.
You also might want to clear any VirtualBox host only interfaces you are not using."

To be honest I just stumbled across this project; was hoping to use it to run some integration tests, but stumbled on the first hurdle. Is there any plans (is it even possible) to make this compatible with net core?