Coder Social home page Coder Social logo

actionunit's People

Contributors

dakusui avatar dependabot[bot] avatar xjj59307 avatar

Stargazers

 avatar  avatar

Watchers

 avatar

actionunit's Issues

Support hooking exceptions generated by retry action

Retry action would swallow all of the exceptions except for the one generated by the last execution. As a result, we would lose all of the intermediate error information. Moreover, if a long-running action is being executed repeatedly, we couldn't see any live information until the whole retry action finishes. Supporting exception hook could help us resolve the above two issues.

Introduce a "tests" action

A "tests" action is desired.

  Action action = tests(
     leaf("test1", c -> {}),
     leaf("test2", c -> {}),
     leaf("test3", c -> {}))

This action will perform its children one by one, however even if any of them fails it doesn't abort the rest.
Yet, the entire test action will fail after all of them are tried.
Only if all of them pass, the action will succeed.

Allow to give Supplier<Stream<T>> to ForEachOf action

It is awkward to force users to implement an Iterable that delays evaluations of element in a stream. Instead we should be able to write this

  $.forEachOf("summary text", () -> readListFromDatabase()).perform("print", System.out::println)

Make it possible to use 'default value' in action creatioin time for 'ForEach' action

I want to be able to do this.

  @PerformWith(Test.class)
  public Action composeSingleLoop2() {
    return sequential(
        simple("print hello", () -> System.out.println("hello")),
        forEachOf(
            asList("A", "B", "C")
        ).withDefault(
            "unknown"
        ).concurrently(
        ).perform(
            (Context $, DataHolder<String> value) -> {
              String v = value.isPresent() ?
                  value.get() :
                  String.format("(%s)", value.get());
              return $.sequential(
                  $.simple("print the given value(1st time)", () -> System.out.println(v)),
                  $.simple("print the given value(2nd time)", () -> System.out.println(v)),
                  $.sleep(2, MICROSECONDS)
              );
            }
        ),
        simple("print bye", () -> System.out.println("bye"))
    );
  }

Support specific multiplicity for concurrent action

Concurrent action is using parallelStream to implement parallel executions of children actions. The default multiplicity of parallelStream is the number of cpu -1 according to this answer. On some occasions, we would like to control the multiplicity like ExecutorService especially for IO intensive actions.

Scopes of several methods in Commander are too narrow

Methods defined in Commander class are defined too narrow.
For instance, shell() method is defined packaged private, while it needs to be called to create a string representation of it.

Also it should implement a Cloneable interface to be able to be cloned when multiple actions should be created from it.

Attempt and forEachOf doesn't support external ActionFactory

The reusability of actions is one of the most important basis of actionunit. However, at least Attempt and forEachOf don't support external ActionFactory. As a result, we couldn't reuse current ActionFactory to create actions for them.

Action action = attempt(simple("attempt", () -> { throw new RuntimeException("attempt"); }))
    .recover(RuntimeException.class, ($, e) -> simple("recover", () -> System.out.println("recover")))
    .ensure(($) -> simple("ensure", () -> System.out.println("ensure")));
[x]Attempt
Exception in thread "main"   [x]Target
    [x]attempt
  [x]Recover(RuntimeException)
    []recover
  [x]Ensure
    []ensure
java.lang.IllegalStateException: Node matching '5(ensure)' was not found under '2(Ensure)'(3-ensure)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.lambda$toNode$3(ReportingActionPerformer.java:125)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.toNode(ReportingActionPerformer.java:123)
	at com.github.dakusui.actionunit.visitors.ActionWalker.handle(ActionWalker.java:178)
	at com.github.dakusui.actionunit.visitors.ActionWalker.visit(ActionWalker.java:27)
	at com.github.dakusui.actionunit.visitors.ActionPerformer.visit(ActionPerformer.java:20)
	at com.github.dakusui.actionunit.actions.Leaf.accept(Leaf.java:36)
	at com.github.dakusui.actionunit.visitors.ActionWalker.lambda$namedActionConsumer$0(ActionWalker.java:144)
	at com.github.dakusui.actionunit.visitors.ActionWalker.handle(ActionWalker.java:184)
	at com.github.dakusui.actionunit.visitors.ActionWalker.visit(ActionWalker.java:38)
	at com.github.dakusui.actionunit.visitors.ActionPerformer.visit(ActionPerformer.java:20)
	at com.github.dakusui.actionunit.actions.Named$Impl.accept(Named.java:63)
	at com.github.dakusui.actionunit.visitors.ActionPerformer.lambda$attemptActionConsumer$10(ActionPerformer.java:95)
	at com.github.dakusui.actionunit.visitors.ActionWalker.handle(ActionWalker.java:184)
	at com.github.dakusui.actionunit.visitors.ActionWalker.visit(ActionWalker.java:102)
	at com.github.dakusui.actionunit.visitors.ActionPerformer.visit(ActionPerformer.java:20)
	at com.github.dakusui.actionunit.actions.Attempt$Impl.accept(Attempt.java:94)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.perform(ReportingActionPerformer.java:70)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.performAndReport(ReportingActionPerformer.java:63)
	at com.github.xjj59307.Foo.main(Foo.java:78)

Enable to override variables

Current Context does not allow to override variables following code. Overridable variable is more useful for user.

Context.java

    public Context assignTo(String variableName, Object value) {
      if (this.variables.containsKey(requireNonNull(variableName)))
        throw new RuntimeException();
      this.variables.put(variableName, value);
      return this;
    }

Make it possible to define a scope by users

Not relying on actionunit framework to create a child context implicitly, maybe we should let users define a scope by themselves.

  public static Action scope(List<String> localVariables, Action action) {

  }

Retry prints x as many as retry times

Action action = ActionSupport.retry(ActionSupport.simple("attempt", () -> { throw new RuntimeException("attempt"); }))
    .withIntervalOf(100, TimeUnit.MILLISECONDS)
    .times(50)
    .on(RuntimeException.class)
    .build();
[x]Retry(100[milliseconds]x50times)
  [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]attempt

As we can see, retry has printed out 50 times x which is a little noisy if the retry times is a large number.

Make it possible to use variables within actions

I want to be able to do following.

  public void example() {
    Context top = new Context.Impl();

    run(
        top.sequential(
            top.simple("init", () -> top.set("i", 0)),
            top.forEachOf(asList("a", "b", "c", "d", "e", "f"))
                .perform(
                    ($, data) ->
                        $.sequential(
                            $.simple(
                                "print i",
                                () -> System.out.printf("%d %s%n", top.<Integer>get("i"), data.get())
                            ),
                            $.simple(
                                "i++",
                                () -> top.set("i", top.<Integer>get("i") + 1)
                            ))))
    );
  }

  private void run(Action action) {
    ReportingActionPerformer performer = ReportingActionPerformer.create(action);
    performer.report();
    performer.performAndReport();
  }
```
, which results in the output below.

```
[]Sequential (2 actions)
  []init
  []ForEach (SEQUENTIALLY) [a, b, c, d, e, f]
    []Sequential (2 actions)
      []print i
      []i++
0 a
1 b
2 c
3 d
4 e
5 f
[o]Sequential (2 actions)
  [o]init
  [o]ForEach (SEQUENTIALLY) [a, b, c, d, e, f]
    [o...]Sequential (2 actions)
      [o...]print i
      [o...]i++
```

Support fast-fail for concurrent action

The whole concurrent action won't finish until all its children actions finish even one of them may have already failed. This isn't expected when we would like to see a fast-fail on the action. We should support both behavior for concurrent action.

Action action = ActionSupport.concurrent(
  ActionSupport.simple("", () -> { throw new RuntimeException(); }),
  ActionSupport.sleep(10, TimeUnit.SECONDS)
);

action fails after 10 seconds.

Create a feature to use CLI programs as actions

I want to be able to do something like this.

  Function<Context, Action> createFiles() {
    return (Context $) -> $.named(
        "prepare test file",
        $.sequential(
            new Touch($).cwd(dir).add("a").build(),
            new Touch($).cwd(dir).add("b").build(),
            new Touch($).cwd(dir).add("c").build()
        ));
  }

Think of simplifying the interface of whilst and when actions

The current interface of whilst and when actions provided by ActionSupport is as follows.

whilst(Supplier<T> value, Predicate<T> condition);

when(Supplier<T> value, Predicate<T> condition);

I would say under most simple scenarios, we wouldn't separate the logic of Supplier<T> value and Predicate<T> condition. As a result, we have to put a placeholder like whilst(condition, (input) -> input). We could provide another interface like whilst(Supplier<Boolean> value).

Support jumphost option in SshOption

In the SshOptions class, it would be useful to be able to add jumphost proxies (more info on ProxyJump).

By supporting this, it would be simpler to support connecting to remote servers that require jumping through a jumpbox or firewall.

Improve tree formatting

Improve tree formatting on "ForEach" or "When/Otherwise" structure.

If we do something like following,

    Integer          boundary = 100;
    ContextPredicate cp       = ContextPredicate.of("j",
        predicate((Integer x) -> Objects.equals(x, 0)).describe("{0}==0")
            .or(predicate((Integer x) -> x > 0).describe("{0}>0"))
            .and(predicate((Integer x) -> x < boundary).describe(() -> "{0}<" + boundary)
            )).negate();

    ContextConsumer cc = contextConsumerFor("i").with(
        consumer(System.out::println).describe("System.out::println")
    ).andThen(contextConsumerFor("j").with(
        consumer(System.err::println).describe("System.err::println")
    ));

    @Test
    public void whenPerformedNestedLoop$thenWorksCorrectly() {
      ReportingActionPerformer.create(Writer.Std.OUT).performAndReport(
          forEach("i", c -> Stream.of("Hello", "world"))
              .perform(
                  forEach("j", c -> Stream.of(-1, 0, 1, 2, 100)).perform(
                      when(cp)
                          .perform(leaf(cc))
                          .otherwise(nop())
                  )));
    }

We'd get output like this.

Hello
-1
Hello
100
world
-1
world
100
[o]for each of data sequentially
  [oo]for each of data sequentially
    [o...]if [!j:[((j==0||j>0)&&j<100)]] is satisfied
      [o...]then
        [o...]i:[System.out::println];j:[System.err::println]
      [o...]else
        [o...](nop)

Simplify action tree reported by ReportingActionPerformer

I would see the following output:

[E:0]for each of (noname) parallely
  [EE:0]do sequentially
  +-[EE:0]print1
  |   [EE:0](noname)
  +-[]print2
  |   [](noname)
  +-[]do sequentially
    +-[]print2-1
    |   [](noname)
    +-[]print2-2
        [](noname)

for a Java program:

public class Example extends TestUtils.TestBase {
  @Ignore
  @Test
  public void test() {
    Action action = forEach(
        "i",
        (c) -> Stream.of("hello", "world")
    ).parallelly(
    ).perform(
        sequential(
            simple("print1", (c) -> System.out.println(v(c))),
            simple("print2", (c) -> System.out.println(v(c))),
            sequential(
                simple("print2-1", (c) -> System.out.println(v(c))),
                simple("print2-2", (c) -> System.out.println(v(c)))
            )
        )
    );

    ReportingActionPerformer.create().performAndReport(action, Writer.Std.OUT);
  }

  static private String v(Context c) {
    return c.valueOf("v");
  }
}

Allow "clone" for actions

We need to allow "clone" operation on an action, otherwise our users need to create a logic to do so everytime when they want to reuse existing action twice in the same action tree.

Reusing an action is possible without allowing this operation, though.
However, the reused action will appear in reports with incorrect information.

Warnings are given on release:prepare

Following warnings are given when mvn release:{prepare,perform} is executed.

[INFO] Executing goals 'deploy'...
[WARNING] Maven will be executed in interactive mode, but no input stream has been configured for this MavenInvoker instance.
[INFO] [INFO] Scanning for projects...
[INFO] [WARNING] 
[INFO] [WARNING] Some problems were encountered while building the effective model for com.github.dakusui:actionunit:jar:5.1.4
[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-source-plugin is missing.
[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-javadoc-plugin is missing.
[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing.
[INFO] [WARNING] 
[INFO] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[INFO] [WARNING] 
[INFO] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[INFO] [WARNING] 
[INFO] [INFO] 
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Building actionunit 5.1.4
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] 
[INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ actionunit ---
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] [INFO] skip non existing resourceDirectory /home/hiroshi/workspace/actionunit/target/checkout/src/main/resources
[INFO] [INFO] 
[INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ actionunit ---
[INFO] [INFO] Changes detected - recompiling the module!
[INFO] [INFO] Compiling 36 source files to /home/hiroshi/workspace/actionunit/target/checkout/target/classes
[INFO] [INFO] 
[INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ actionunit ---
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] [INFO] skip non existing resourceDirectory /home/hiroshi/workspace/actionunit/target/checkout/src/test/resources
[INFO] [INFO] 
[INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ actionunit ---
[INFO] [INFO] Changes detected - recompiling the module!
[INFO] [INFO] Compiling 37 source files to /home/hiroshi/workspace/actionunit/target/checkout/target/test-classes
[INFO] [WARNING] /home/hiroshi/workspace/actionunit/target/checkout/src/test/java/com/github/dakusui/actionunit/extras/cmd/CmdExample.java: /home/hiroshi/workspace/actionunit/target/checkout/src/test/java/com/github/dakusui/actionunit/extras/cmd/CmdExample.java uses unchecked or unsafe operations.
[INFO] [WARNING] /home/hiroshi/workspace/actionunit/target/checkout/src/test/java/com/github/dakusui/actionunit/extras/cmd/CmdExample.java: Recompile with -Xlint:unchecked for details.
[INFO] [INFO] 
[INFO] [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ actionunit ---
[INFO] [INFO] Surefire report directory: /home/hiroshi/workspace/actionunit/target/checkout/target/surefire-reports
[INFO] 

ForEachTest#givenConcurrentForEachAction$whenPerformWithReporting$worksCorrectly fails intermittently

It fails about once out of 10 times with spitting following output.


<!>
<Hello>
<world>
[o]ForEach
  [ooo]Sequential (2 actions)
    [ooo]print {s}

java.lang.NullPointerException
	at java.util.LinkedList$ListItr.next(LinkedList.java:893)
	at java.lang.Iterable.forEach(Iterable.java:74)
	at com.github.dakusui.actionunit.visitors.reporting.Report$Record$Formatter$1.formatRecord(Report.java:65)
	at com.github.dakusui.actionunit.visitors.reporting.Report$Record$Formatter$1.format(Report.java:58)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.lambda$perform$34(ReportingActionPerformer.java:59)
	at com.github.dakusui.actionunit.visitors.reporting.Node.walk(Node.java:35)
	at com.github.dakusui.actionunit.visitors.reporting.Node.lambda$walk$32(Node.java:39)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
	at com.github.dakusui.actionunit.visitors.reporting.Node.walk(Node.java:39)
	at com.github.dakusui.actionunit.visitors.reporting.Node.lambda$walk$32(Node.java:39)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
	at com.github.dakusui.actionunit.visitors.reporting.Node.walk(Node.java:39)
	at com.github.dakusui.actionunit.visitors.reporting.Node.walk(Node.java:31)
	at com.github.dakusui.actionunit.visitors.reporting.ReportingActionPerformer.perform(ReportingActionPerformer.java:56)
	at com.github.dakusui.actionunit.ut.actions.ForEachTest.givenConcurrentForEachAction$whenPerformWithReporting$worksCorrectly(ForEachTest.java:74)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:58)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)


No descriptive error message from ActionSupport.TimeOut

This action throws ActionTimeOutExecption when timeout happens. Then, the message is null. This should allow descriptive error messages.

The sample class to reproduce this symptom.


public class ActionTimeoutTest {
	public static void main(String[] args) {
		List<String> alphabets = new ArrayList<String>() {{
			this.add("a");
			this.add("b");
		}};

		Action action = parallel(alphabets.stream()
				.map(alphabet -> leaf(c -> {
					if(alphabet.equals("b")) {
						try {
							Thread.sleep(2000);
						} catch (InterruptedException e) {
						}
					}
				}))
				.map(a -> timeout(a).in(1, SECONDS))
				.map(a -> retry(a)
						.on(ActionException.class)
						.withIntervalOf(1, SECONDS)
						.times(2)
						.$())
				.collect(toList()));

		try {
			ActionUtils.processAction(action, Writer.Slf4J.INFO);
		}catch (ActionTimeOutException e) {
			System.out.println(e.getMessage());
		}
	}
}

Improve report produced by ReportingActionPerformer

Improve report produced by ReportingActionPerformer so that important actions are written with higher log level.
An action marked E/F should be considered more important than passing ones.
Passing ones belonging to a sequential/parallel action one of whose children is failing should be considered more important. (Siblings of a failing action)

Printability improvement

String.format("%s", action)

results in Object#toString which is not readable enough.
Probably implementing toString method in objects returned by methods in ActionSupport will be useful.

A test where given cmd action fails twice and passes, whenRetryOnce$thenFail(RetryOnTimeOutTest) is not passing in Travis CI

Following is output.

whenRetryOnce$thenFail(com.github.dakusui.actionunit.extras.cmd.RetryOnTimeOutTest)  Time elapsed: 4.014 sec  <<< ERROR!
fail
fail
success
[o]do sequentially
  [o](nop)
  [o]attempt
    [o]retry 3 times in 1[seconds] on ActionException
      [EEo]timeout in 1[seconds]
        [EEo](commander)
          [EEo](noname)
    []recover
      []rethrow
        [](noname)
    [o]ensure
      [o](commander)
        [o](noname)
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.049 sec <<< FAILURE!
java.lang.Exception: Unexpected exception, expected<com.github.dakusui.actionunit.exceptions.ActionException> but was<org.junit.ComparisonFailure>

This gives following assertion message and it implies 'cmd' might not be consuming stdout as designed.

Caused by: org.junit.ComparisonFailure: expected:<[and:[
  @size[]->castTo[Integer](x) equalTo[2]
  @get[0](x) equalTo[fail]
  @get[1](x) equalTo[fail]
]]> but was:<[when x=<[fail]>; then and:[
  @size[]->castTo[Integer](x) equalTo[2] was not met because @size[]->castTo[Integer](x)=<1>
  @get[0](x) equalTo[fail]
  @get[1](x) equalTo[fail] failed with java.lang.RuntimeException(java.lang.reflect.InvocationTargetException)
]->false

Even if this is a real bug it should be fixed in 'cmd' side.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.