microsoft / mwt-ds-explore-java

Exploration Library in Java

Exploration Library

The exploration library addresses the ‘gathering the data’ aspect of machine learning rather than the ‘using the data’ aspect we are most familiar with. The primary goal here is to enable individuals (i.e. you) to gather the right data for using machine learning for interventions in a live system based on user feedback (click, dwell, correction, etc…). Empirically, gathering the right data has often made a substantial difference. Theoretically, we know it is required to disentangle causation from correlation effectively in general.

First version of client-side exploration library that includes the following exploration algorithms:

• Epsilon Greedy
• Tau First
• Softmax
• Bootstrap
• Generic (users can specify custom weight for every action)

This release supports Java. For other languages, see: https://github.com/multiworldtesting/.

For sample usage, see: https://github.com/multiworldtesting/explore-java/blob/master/src/main/java/com/mwt/sample/ExploreOnlySample.java

For more details on Multiworld Testing as a Service, visit http://aka.ms/mwt

mwt-ds-explore-java's Issues

Refactor code to avoid random access where it's unnecessary

There are a lot of places where java.util.List iteration follows the pattern:

java.util.List<X> lst = ...
for (int i = 0; i < lst.size; i++) {
  doSomething( lst.get(i) );
}

In most or all situations, these can be replaced with for each-style iteration. Changing this isn't just for stylistic reasons. It can actually have implications on performance guarantees too. Specifically, there are places in the code base like in com/mwt/explorers/BootstrapExplorer.java where you see the following code:

public class BootstrapExplorer<T> implements Explorer<T>, ConsumePolicies<T> {
  private List<Policy<T>> policies;

  public DecisionTuple chooseAction(long saltedSeed, T context) {
    for (int currentBag = 0; currentBag < policies.size(); currentBag++) {
      actionFromBag = policies.get(currentBag).chooseAction(context);  // line 61
    }
  }
}

The problem is on line 61, actionFromBag = policies.get(currentBag).chooseAction(context). Since there's no random access guarantee at an API level for policies because it's a java.util.List, this loop has a worst case runtime of _O(_P ²) rather than the desired _O(_P) runtime.

By changing standard C-style for loops to for each-style loops, we can make tighten the runtime guarantees while not changing the MWT APIs.

Create a release process to release to maven central

It would be nice to have this artifact available on maven central. I could help with this if you would like.

GSON not in pom.xml

I found two issues when trying to compile the project using maven. The first is that GSON is required in com.mwt.tests.BlackBox but is not in the pom.xml file. A second issue that is com.mwt.tests.BlackBox uses java 1.7 classes. To fix these, I would put the com.mwt.tests package in src/test/java and I would update the pom.xml to have the dependencies:

<dependencies>
...
    <dependency>
      <groupId>commons-io</groupId>
      <artifactId>commons-io</artifactId>
      <version>2.4</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>com.google.code.gson</groupId>
      <artifactId>gson</artifactId>
      <version>2.4</version>
      <scope>test</scope>
    </dependency>
...
</dependencies>

This way, GSON is not a runtime dependency of the library since it's only used in the test code.

Additionally, I'd change com.mwt.tests.BlackBox to import

1. org.apache.commons.io.IOUtils
2. java.io.File

Then I would remove the java.nio imports and I would change the first line in the try inside the main method to

String content = IOUtils.toString(new File(args[0]).toURI());

This would fix everything and put tests in the proper subdirectory according to maven convention. I'd be happy to submit a pull request if you'd like.

This repo is missing a LICENSE file

This repository is currently missing a LICENSE file.

A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license Microsoft uses at: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.

If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license (Microsoft-internal guidance).