microsoft / coyote-samples

Get started with Coyote with these simple introductory sample projects.

Coyote samples

THIS REPO IS ARCHIVED AND DOES NOT ACCEPT PULL REQUESTS.

The Coyote samples that were formerly located in this repository have now moved to the main Coyote repository that is located at https://github.com/microsoft/coyote. We welcome you to visit us there to try out Coyote, join our community and contribute!

coyote-samples's Issues

Failing to resolve assemblies on macOS

I'm trying to run the sample app ImageGalleryAspNet and I'm getting this reference error:

$ coyote rewrite rewrite.coyote.json
Microsoft (R) Coyote version 1.3.0.0 for .NET Core 5.0.0
Copyright (C) Microsoft Corporation. All rights reserved.

. Rewriting the assemblies specified in rewrite.coyote.json
... Copying all files to the '/Users/cederstrom/dev/labb/coyote/coyote-samples/WebApps/ImageGalleryAspNet/bin/coyote' directory
... Rewriting the 'ImageGalleryService.dll' assembly (ImageGalleryService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null)
Unable to resolve assembly: Microsoft.Extensions.Hosting.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
Failed to resolve assembly: 'Microsoft.Extensions.Hosting.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
... Rewriting the 'ImageGalleryTests.dll' assembly (ImageGalleryTests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null)
Unable to resolve assembly: Microsoft.Extensions.Logging.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
Failed to resolve assembly: 'Microsoft.Extensions.Logging.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
... Rewriting the 'TraceReplayer.dll' assembly (TraceReplayer, Version=1.3.0.0, Culture=neutral, PublicKeyToken=null)
... Writing the modified 'TraceReplayer.dll' assembly to /Users/cederstrom/dev/labb/coyote/coyote-samples/WebApps/ImageGalleryAspNet/bin/coyote/TraceReplayer.dll
. Done rewriting in 0.427981 sec

Am I missing something?

My dotnet info:

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.302
 Commit:    c005824e35

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.302/

Host (useful for support):
  Version: 5.0.8
  Commit:  35964c9215

.NET SDKs installed:
  2.1.810 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]
  3.1.102 [/usr/local/share/dotnet/sdk]
  3.1.201 [/usr/local/share/dotnet/sdk]
  3.1.301 [/usr/local/share/dotnet/sdk]
  3.1.401 [/usr/local/share/dotnet/sdk]
  3.1.402 [/usr/local/share/dotnet/sdk]
  3.1.403 [/usr/local/share/dotnet/sdk]
  5.0.100 [/usr/local/share/dotnet/sdk]
  5.0.302 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.22 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.22 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0-rc.2.20475.17 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.21 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.22 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0-rc.2.20475.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Cannot run the binaries with coyote.exe

I was trying to run some test here by following the steps on https://microsoft.github.io/coyote/learn/tutorials/failover-coffee-machine-tasks

However, I failed to run the binaries with coyote.exe, I kept getting errors like:

.\coyote test ../netcoreapp3.1/CoffeeMachineTasks.dll -i 100 -ms 2000 --sch-pct 10
. Testing ../netcoreapp3.1/CoffeeMachineTasks.dll
Starting TestingProcessScheduler in process 48028
Error: Failed to get test method '' from assembly 'CoffeeMachineTasks, Version=1.0.6.0, Culture=neutral, PublicKeyToken=null'

I tried the other tests. And it looks like all the Executes can't be started by coyote.exe because of this error.

Update Global.json

I was attempting to load the helloworldactors and the project failed to load, complaining about the .net version.

I looked in the main coyote repo and saw that the global.json specified
{
"sdk": {
"version": "3.1.401"
}
}
Where as the samples global.json specified
{
"sdk": {
"version": "3.1.300"
}
}
Once I changed to the latest version, the project loaded and compiled without incident.

Is there supposed to be a bug in CloudMessaging/Raft.Nondeterminism?

I cloned the repo and ran the CloudMessaging/Raft.Nondeterminism sample:

PS C:\Users\dsprague\source\repos\coyote-samples> coyote.exe test .\bin\netcoreapp3.1\Raft.Nondeterminism.dll -i 10000 -ms 20 --coverage activity


. Testing .\bin\netcoreapp3.1\Raft.Nondeterminism.dll
... Started the testing task scheduler (process:42224).
... Created '1' testing task (process:42224).
... Task 0 is using 'random' strategy (seed:2599116392).
... Telemetry is enabled, see http://aka.ms/coyote-telemetry.
..... Iteration #1
[...]
... Testing statistics:
..... Found 0 bugs.
... Scheduling statistics:
..... Explored 10000 schedules: 10000 fair and 0 unfair.
..... Number of scheduling points in fair terminating schedules: 200 (min), 200 (avg), 200 (max).
..... Exceeded the max-steps bound of '20' in 100.00% of the fair schedules.
... Elapsed 104.7570836 sec.

As you can see, it didn't detect any bugs. This matches my understanding of the Raft protocol, that it's tolerant of duplicate messages, but the documentation seems to indicate that coyote test should detect a bug. From https://microsoft.github.io/coyote/learn/tutorials/raft-mocking:

You can also explore the Raft.Nondeterminism.dll version of this sample that injects a bug in the system by randomly sending duplicate VoteRequestEvents. Then you can see how the coyote test tool is able to spot the resulting bugs because of the SafetyMonitor.

Should coyote test find a bug in this example, or not?

Monitors/FailureDetector Example in readme has incorrect name and type

the readme in Monitors readme.md

To test for the bug execute the following command:

coyote test FailureDetector.exe -i 500000 --max-steps 200

No FailureDetector assemblies are produced. Also this should be .dll not .exe to succeed.

<Project Sdk="Microsoft.NET.Sdk">
  <Import Project="..\Common\build.props" />
  <PropertyGroup>
    <Description>The Monitors sample.</Description>
    <AssemblyName>Monitors</AssemblyName>
    <RootNamespace>Microsoft.Coyote.Samples.Monitors</RootNamespace>
    <OutputType>Exe</OutputType>
    <OutputPath>..\bin\</OutputPath>
  </PropertyGroup>
</Project>

changing the line to

coyote test Monitors.dll -i 500000 --max-steps 200

works.

Hello World Actors

The assert in the HandleGreeting goes off because decrementing the global Count variable throws off the for loop.

In reality these two variables would never be related, let alone the same, IMO.

Fixing the bug with leader election in Raft example - is this desired?

As documented, the Raft example has a bug with leader election, which can be discovered by running coyote test with the mock in Raft.Nondeterminism. This bug appears to be due to the way votes for a leader are counted; if messages from the same node are duplicated, they'll be registered as separate votes by Server.VoteResponse(). This bug can be fixed by tracking which nodes votes are coming from; however, I'm not sure if it's desirable to keep this bug around for demonstrating bugfinding. What's the maintainers' point of view on this?