🍰 A property-based testing framework that tries to break your invariances.
License: MIT License
I'm Marcel Garus (he/him), a student at the 
In the open-source world, I work mostly on low-level things such as Rust, Assembly, or Zig. I love inventing new programming languages such as Candy, Martinaise, Soil, Mehl, Dungbeetle. Sometimes, I write articles on my blog.
📫 Don't hesitate to get in touch with me! You can reach me on Telegram or via other contact options.
How to use this package for model based testing?
Example: https://github.com/dubzzz/fast-check/blob/main/documentation/Tips.md#model-based-testing-or-ui-test
My current implementation:
import 'package:glados/glados.dart';
void main() {
Glados<List<Command<IntModel, IntList>>>(any.nonEmptyList(any.command))
.test('formatCountdown', (commands) {
// print(commands);
final model = IntModel(0);
final real = IntList();
for (final command in commands) {
if (command.check(model)) {
command.run(model, real);
}
}
});
}
class IntList {
final data = <int>[];
void push(int v) {
data.add(v);
}
int? pop() {
if (data.isEmpty) {
return null;
}
final lastElement = data.last;
data.removeLast();
return lastElement;
}
int size() => data.length;
}
class IntModel {
late int num;
IntModel(this.num);
}
abstract class Command<Model, Real> {
bool check(Model model);
void run(Model model, Real real);
}
class PushCommand implements Command<IntModel, IntList> {
late int value;
PushCommand(this.value);
bool check(IntModel m) => true;
void run(IntModel m, IntList r) {
r.push(value); // impact the system
m.num++; // impact the model
}
String toString() => 'push($value)';
}
class PopCommand implements Command<IntModel, IntList> {
bool check(IntModel m) {
// should not call pop on empty list
return m.num > 0;
}
void run(IntModel m, IntList r) {
expect(r.pop() is int, true);
m.num--;
}
String toString() => 'pop';
}
class SizeCommand implements Command<IntModel, IntList> {
bool check(IntModel m) => true;
void run(IntModel m, IntList r) {
expect(r.size(), m.num);
}
String toString() => 'size';
}
extension AnyCommand on Any {
Generator<Command<IntModel, IntList>> get command {
return (random, size) {
return ShrinkableCombination(<Shrinkable<int>>[
any.int(random, size),
], (List<dynamic> ints) {
final commands = [
...ints.map((v) => PushCommand(v as int)).toList(),
PopCommand(),
SizeCommand()
];
return commands[random.nextInt(commands.length)];
});
};
}
}Hi @MarcelGarus,
could you release a new version that includes #22 ?
I just ran into this and wanted to create a similar PR 😉
Thank you for creating this package!
OS: Windows 11
IDE: IntelliJ Idea
Dart SDK: '>=2.17.6 <3.0.0'
Dependencies:
#dependencies:
dev_dependencies:
glados: ^1.1.3
lints: ^2.0.0
test: ^1.16.0I've been practicing property-based testing with this library and I've run into a few instances of tests taking a very long time to run and eventually failing with exit code -1.
Here is an implementation of division where I purposefully return multiplication to test division's properties:
double div(double num1, double num2) => num1 * num2;and my only test is:
Glados<double>().test('div', (num1) => expect(div(num1, num1), 1));I initially thought that the test was hanging but it actually took 6 minutes to run and eventually failed without telling me on what values. It works fine when I change num1 * num2 to num1 / num2 so I believe the issue has something to do with glados' shrinking in regards to doubles since the following code failed quickly without issues:
double div(int num1, int num2) => (num1 * num2).toDouble();Glados<int>().test('div2', (num1) => expect(div(num1, num1), 1));Is there a way to run setUp and tearDown per test iteration?
Glados allows you to choose from only two generators:
any.either(subZeroGenerator, scorpionGenerator);Add more generic way to select from multiple generators
extension AnyUtils on Any {
...
Generator<T> chooseGenerator<T>(List<Generator<T>> generators) {
return (random, size) {
final chosenGenerator = choose(generators)(random, size).value;
return chosenGenerator(random, size);
};
}
...
}any.chooseGenerator([subZeroGenerator, scorpionGenerator, liuKangGenerator]);@MarcelGarus I'm having an issue with upgrading again. I'm relatively certain I had filed something similar before but I can't seem to find the issue any more. Not as urgent this time, at least I don't have to remove the library for the time being, I only cannot upgrade.
Thanks for looking into this, and for your work on this useful library.
Because test >=1.20.0 depends on test_api 0.4.9 and test >=1.19.4 <1.20.0 depends on test_api 0.4.8, test >=1.19.4 requires test_api 0.4.8 or 0.4.9.
And because glados >=1.1.2 depends on test ^1.19.5 and every version of flutter_test from sdk depends on test_api 0.4.3, glados >=1.1.2 is incompatible with flutter_test from sdk.
So, because lotti depends on both flutter_test from sdk and glados ^1.1.2, version solving failed.
Running "flutter pub get" in lotti...
pub get failed (1; So, because lotti depends on both flutter_test from sdk and glados ^1.1.2,
version solving failed.)
Hi! The title says it all, but here's the long story. First of all, thank you very much for writing this neat package :) I encountered it completely by chance but it got me into exploring different approaches to testing code and essentially made me appreciate writing tests instead of viewing them as a mostly tedious, repetitive chore.
I encountered one quality of life issue that's been bugging me for the past few days: I use vscode with the dart-code extension and the editor gets all confused when trying to launch a single Glados test using vscode's gutter button and code lens (see screenshot). That's because it searches for the test by name and fails with the message below.
// output for a test defined as `Glados(any.int).test('description goes here', (_) {});`
No tests match regular expression "^description goes here( \(variant: .*\))?$".
Exited (1)
The extension expects to match the name exactly, with an optional (variant: <whatever>) suffix, while Glados appends a (testing $numRuns inputs) string. I'm not sure if it would even be possible for the extension to resolve this kind of a dynamic name.
So, here's my request: would it be ok to drop the '(testing x inputs)' suffix? Instead, the test could print this information.
The isTest annotation docs also sound to me as if the test description was supposed to be passed around unmodified.
Used to annotate a test framework function that runs a single test.
Tools, such as IDEs, can show invocations of such function in a file structure view to help the user navigating in large test files.
The first parameter of the function must be the description of the test.
I'm not sure how this affects other IDEs, though, since I mainly use vscode.
What do you think? Would you mind making this change? If it's fine I could prepare a pull request.
Can I use glados with Widget Test?
If not, do you plan to support it?
See #4 for more details.
This issue is not an immediate priority for me. If you need this feature, don't hesitate to ping me here.
First of all: Great Project! Unfortunately I encountered a little bug.
The provided ExploreConfig explore property in the Glados2 and Glados3 implementation has no influence.
I use Glados version 1.1.5 .
As far I can see it, you did miss to forward the property to the Glados constructors in line 256 and 331 to make it work correctly.
BTW. Nice university, I also studied there some years ago! :) .
Unfortunately, Glados is currently incompatible with the most recent version of many libraries
Glados 1.1.7 depends on meta 1.11.0 but Flutter 3.16.x has a pinned dependency on meta 1.10.0
Is there any workaround when you need 4 inputs, or should I create a Glados4 class and submit a pull request?
Hi Marcel,
could you tell more about the background of the library, its theoretical foundations, its architecture, its history, how it came to be...
That would be very interesting.
Kind regards!
The license listed for this repository isn't recognized as an OSI-approved license on GitHub or on pub.dev.
Updating this license to an OSI-approved one like BSD-3 Clause would make users such as myself much more comfortable with using this package and would also increase your pub points on pub.dev (it is currently penalized 10 points because of this issue).
The range of integers generated by fixed-length integer generators does not match the documentation, or an assertion error occurs. For example, any.uint8 is supposed to generate integers ranging from 0 to 255 inclusive, but in reality, it generates integers ranging from 0 to 511 inclusive.
The affected generators include:
any.uint8any.uint16any.uint32any.int8any.int16any.int32any.int64 (causes an assertion error)any.uint8 generates integers from 0 (inclusive) to 512 (exclusive).any.uint16 generates integers from 0 (inclusive) to 131072 (exclusive).any.uint32 generates integers from 0 (inclusive) to 8589934592 (exclusive).any.int8 generates integers from -256 (inclusive) to 256 (exclusive).any.int16 generates integers from -65536 (inclusive) to 65536 (exclusive).any.int32 generates integers from -4294967296 (inclusive) to 4294967296 (exclusive).any.int64 is causing an assertion error.The assertion error caused by using any.int64 is as follows:
00:00 +0 -1: loading test/glados_report_test.dart [E]
Failed to load "test/glados_report_test.dart": 'package:glados/src/anys.dart': Failed assertion: line 27 pos 44: 'min < max': is not true.
dart:core _AssertionError._throwNew
package:glados/src/anys.dart 27:44 IntAnys.intInRange
package:glados/src/anys.dart 81:36 IntAnys.int64
test/glados_report_test.dart 30:14 main
The range of integers generated by the generators matches the documentation.
any.uint8 generates integers from 0 (inclusive) to 256 (exclusive).any.uint16 generates integers from 0 (inclusive) to 65536 (exclusive).any.uint32 generates integers from 0 (inclusive) to 4294967296 (exclusive).any.int8 generates integers from -128 (inclusive) to 128 (exclusive).any.int16 generates integers from -32768 (inclusive) to 32768 (exclusive).any.int32 generates integers from -2147483648 (inclusive) to 2147483648 (exclusive).any.int64 generates integers from -9223372036854775808 (inclusive) to 9223372036854775808 (exclusive).The current implementation uses a bit shift of 2 where it should use a bit shift of 1. The assertion error when using any.int64 occurs because the specified minimum value -(2 << 63) and the maximum value 2 << 63 both result in 0. Changing from a bit shift of 2 to a bit shift of 1 should fix the issue.
The current implementation is as follows:
/// A generator that returns [int]s between `0`, inclusive, to `256`, exclusive.
Generator<core.int> get uint8 => intInRange(0, 2 << 8);
/// A generator that returns [int]s between `0`, inclusive, to `65536`,
/// exclusive.
Generator<core.int> get uint16 => intInRange(0, 2 << 16);
/// A generator that returns [int]s between `0`, inclusive, to `4294967296`,
/// exclusive.
Generator<core.int> get uint32 => intInRange(0, 2 << 32);
/// A generator that returns [int]s between `-128`, inclusive, to `128`,
/// exclusive.
Generator<core.int> get int8 => intInRange(-(2 << 7), 2 << 7);
/// A generator that returns [int]s between `-32768`, inclusive, to `32768`,
/// exclusive.
Generator<core.int> get int16 => intInRange(-(2 << 15), 2 << 15);
/// A generator that returns [int]s between `-2147483648`, inclusive, to
/// `2147483648`, exclusive.
Generator<core.int> get int32 => intInRange(-(2 << 31), 2 << 31);
/// A generator that returns [int]s between `-9223372036854775808`, inclusive,
/// to `9223372036854775808`, exclusive.
Generator<core.int> get int64 => intInRange(-(2 << 63), 2 << 63);The tests are also implemented using the same bit shift expressions, so the tests need to be corrected as well.
glados/glados/test/anys_test.dart
Line 39 in d86f83b
Glados(any.uint8).test('uint8', (number) {
expect(number, greaterThanOrEqualTo(0));
expect(number, lessThan(2 << 8));
});
Glados(any.uint16).test('uint16', (number) {
expect(number, greaterThanOrEqualTo(0));
expect(number, lessThan(2 << 16));
});
Glados(any.uint32).test('uint32', (number) {
expect(number, greaterThanOrEqualTo(0));
expect(number, lessThan(2 << 32));
});Unit tests verifying the range of integers generated by the generators are shown below.
import 'package:glados/glados.dart';
void main() {
Glados(any.uint8).test('uint8: 0 to 255', (value) {
expect(0 <= value && value <= 255, true);
});
Glados(any.uint16).test('uint16: 0 to 65535', (value) {
expect(0 <= value && value <= 65535, true);
});
Glados(any.uint32).test('uint32: 0 to 4294967294', (value) {
expect(0 <= value && value <= 4294967295, true);
});
Glados(any.int8).test('int8: -128 to 127', (value) {
expect(-128 <= value && value <= 127, true);
});
Glados(any.int16).test('int16: -32768 to 32767', (value) {
expect(-32768 <= value && value <= 32767, true);
});
Glados(any.int32).test('int32: -2147483648 to 2147483647', (value) {
expect(-2147483648 <= value && value <= 2147483647, true);
});
// This code will fail due to an assertion error
/*
Glados(any.int64).test('int64: -9223372036854775808 to 9223372036854775807',
(value) {
expect(-9223372036854775808 <= value && value <= 9223372036854775807, true);
});
*/
}The test results are as follows:
% dart test --no-fail-fast
00:00 +0: loading test/glados_report_test.dart
intInRange(0, 0);
00:00 +0: test/glados_report_test.dart: uint8: 0 to 255 (testing 100 inputs)
Tested 2 inputs, shrunk 175 times.
Failing for input: 256
00:00 +0 -1: test/glados_report_test.dart: uint8: 0 to 255 (testing 100 inputs) [E]
Expected: <true>
Actual: <false>
package:matcher expect
test/glados_report_test.dart 7:5 main.<fn>
package:glados/src/glados.dart 178:28 Glados.test.<fn>
To run this test again: dart test test/glados_report_test.dart -p vm --plain-name 'uint8: 0 to 255 (testing 100 inputs)'
00:00 +0 -1: test/glados_report_test.dart: uint16: 0 to 65535 (testing 100 inputs)
Tested 2 inputs, shrunk 58799 times.
Failing for input: 65536
00:00 +0 -2: test/glados_report_test.dart: uint16: 0 to 65535 (testing 100 inputs) [E]
Expected: <true>
Actual: <false>
package:matcher expect
test/glados_report_test.dart 11:5 main.<fn>
package:glados/src/glados.dart 178:28 Glados.test.<fn>
To run this test again: dart test test/glados_report_test.dart -p vm --plain-name 'uint16: 0 to 65535 (testing 100 inputs)'
00:00 +1 -2: test/glados_report_test.dart: int8: -128 to 127 (testing 100 inputs)
Tested 1 input, shrunk 77 times.
Failing for input: -129
00:00 +1 -3: test/glados_report_test.dart: int8: -128 to 127 (testing 100 inputs) [E]
Expected: <true>
Actual: <false>
package:matcher expect
test/glados_report_test.dart 19:5 main.<fn>
package:glados/src/glados.dart 178:28 Glados.test.<fn>
To run this test again: dart test test/glados_report_test.dart -p vm --plain-name 'int8: -128 to 127 (testing 100 inputs)'
00:00 +1 -3: test/glados_report_test.dart: int16: -32768 to 32767 (testing 100 inputs)
Tested 2 inputs, shrunk 26031 times.
Failing for input: 32768
00:00 +1 -4: test/glados_report_test.dart: int16: -32768 to 32767 (testing 100 inputs) [E]
Expected: <true>
Actual: <false>
package:matcher expect
test/glados_report_test.dart 23:5 main.<fn>
package:glados/src/glados.dart 178:28 Glados.test.<fn>
To run this test again: dart test test/glados_report_test.dart -p vm --plain-name 'int16: -32768 to 32767 (testing 100 inputs)'
00:08 +1 -4: ... int32: -2147483648 to 2147483647 (testing 100 inputs)
63:09 +1 -4: test/glados_report_test.dart: int32: -2147483648 to 2147483647 (testing 100 inputs)
Tested 1 input, shrunk 2147483648 times.
Failing for input: -2147483649
63:09 +1 -5: test/glados_report_test.dart: int32: -2147483648 to 2147483647 (testing 100 inputs) [E]
Expected: <true>
Actual: <false>
package:matcher expect
test/glados_report_test.dart 27:5 main.<fn>
package:glados/src/glados.dart 178:28 Glados.test.<fn>
To run this test again: dart test test/glados_report_test.dart -p vm --plain-name 'int32: -2147483648 to 2147483647 (testing 100 inputs)'
63:09 +1 -5: Some tests failed.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.