microsoft / praxxis

A task interface for Jupyter notebooks built on machine learning and big data

A Command Line Notebook Task Interface
Explore the docs

Report Bug . Request Feature

Table of Contents

• About the Project
  • Features
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Roadmap
• Contributing
• License
• Contact
• Acknowledgements

About The Project

praxxis is a task interface built on big data and machine learning. Using your own storage pool to collect data on your habits when running notebooks, praxxis will learn about the problems you are solving, correlate that with the problems everyone else is solving, and predict the next notebook you should run without interrupting your workflow. It is a tool based on a collaborative paradigm of problem solving, allowing every person to leverage everyone's knowledge to come to a solution more quickly.

Using praxxis, any command can be run, documented, and reproduced using executable code cells in jupyter notebooks, allowing even the least technical user to jump in right where you left off.

Features

Scenes

praxxis scenes are situation-specific configurations that can be saved, closed, reopened and shared. Scenes store your habits, history, and parameter settings, allowing you to easily fix old problems and get help with new ones. When you share your scenes, your peers are able to see the same outputs, history, and parameter values you see, allowing for easier problem solving in groups.

Predictions

With or without a storage pool, praxxis's predictions are usable through trained machine learning models. If you have your own storage pool, you can top up or train a new model with your own data.

History

Using praxxis, a history of commands is preserved, allowing you to backtrack through problems. Since situation specific configurations are saved as parameters in scenes, you'll always be able to know exactly what commands were run, what was changed, and where you need to go next.

Notebook Libraries

Praxxis runs on libraries of jupyter notebooks, allowing every command on your system to be documented and explained in a useful markdown format. By directly running the code embedded in the documentation, you know that no information is being lost, and no documentaion is getting out of date.

Parameters

Praxxis uses parameter tags to inject parameters into code cells. By saving parameters through praxxis, your environments are saved through sessions and restarts, and are documented in an easily accessible format.

Getting Started

to get started developing or using praxxis, follow these steps.

Prerequisites

• python 3.6 or above

Installation

to install, simply run

pip install praxxis

or, for development mode clone the repo and run

pip install -e .

Usage

praxxis is a command line tool for running jupyter notebooks. To run for the first time, open up your terminal after installing, and run

prax

to see the help page.

For more examples, please refer to the Documentation

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

We would love your help!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Praxxis uses pytest for testing, and it would be much appreciated if you could write tests for your changes before opening a pull request!

We also reference Python PEP-8 for our coding style.

Please see our contributing.md for more details on our coding standards, and code of conduct.

License

Distributed under the MIT License. See LICENSE for more information.

Project Link: https://github.com/microsoft/praxxis

praxxis's Issues

stop and start should be standardized

in some places it's query_start, and query_end, some places it's start and end, and some places it's start and stop and that's.... not very good at all.

ADS can be installed anywhere

we enforce that it's in one of two places, and in hindsight that probably wasn't for the best .

we should add a prompt that allows people to add the path to their install if it's not found in the default place.

import parameters from another scene

allow parameters to be imported into another scene, so that new scenes can be quickly populated with useful params.

run notebook through list of scenes

allow an option in running a notebook to specify any number of scenes to run it through, to allow notebooks to be run quickly on any number of environments.

add display for deleting parameter

right now parameters are deleted silently with no display

Settings portability improvements

-allow multiple settings to be set in one line e.g. "prax u -username user1 -ip some.ip"
-allow for importing a .toml file of settings
-allow for exporting settings to a toml file (? security concern -- discuss)

add running notebooks from file path

allow people to run notebooks by passing in a path to run

HTML open not working on MacOS

both prax o 1 html and prax r 1 html execute without error but do not open an html file on MacOS run in zsh

can't run tests on mac

not sure why!

Library tests breaking in some test orders

When tests are run with --random-order-seed=317856, test_add_regular_library and test_sync_library both fail. I believe the error is in test_add_regular_library, where a LibraryNotFoundError is raised.

This issue is observed on Ubuntu and Windows (Powershell)

prevent sql injection

even though this is a pretty client-side program, the fact that you can just put whatever you want into your sql db is Extremely bad. We need to standardize the sqlite files with variable injection.

duplicate notebook name error

when loading a notebook from a different library that has the same name as another loaded notebook, the duplicate notebook warning still appears. but it still loads the notebook as expected.

to reproduce:
make a copy of test_param_inject.ipynb somewhere else

prax al tests/test_notebooks
prax a test_param_inject.ipynb

the readme is cool but not cool enough

export the terminalizer yaml in the docs folder to improve the readme with gifs.

library ordinal referencing bug

Strange error when loading different notebooks

When I try to open other notebooks besides the ones in the "test" folder, there's always an error like what it shows in the image.

Can you open other random notebooks, or does the error only happen to me?

remove all f-strings

#7
f-strings are cool and I love them, but it also requires that praxxis is on python 3.6 and up, which isn't actually that great.
In order to achieve compatability with Windows XP, (and like debian stable or whatever) we need to remove all of the f-strings from praxxis.

load scene

allow shared scene dbs to be loaded into praxxis, enabling scene sharing.

color doesn't change when viewing set params of a duplicate named notebook

the color of the params should be blue when viewing overridden parameters when viewing a noebook, but they are currently unchanged.

Telemetry options to view errors and send all

two new (very related) features around debugging telemetry problems

things to consider:

• should sending all still be a subprocess, or kept in the console with non-silenced errors?
• should there be a visible clear backlog option that removes files from backlog without sending?

Wording changes for rules addition/deletion

• adding rules is confusing and should be less so

The injected parameters don't take any effect

As can be seen here, I found that when I inject new parameters and try to run the notebook, the injected parameters do not take any effect.

I'm quite stress testing the app, sorry :/
Keep up the good work, the idea behind is very interesting!

environment ordinal bug

prax se with an ordinal fails with a db error

ERROR: json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)

This error appears when I need to open a notebook using jupyter (forcing me to open using vim or html :( ).

---

Specs:
Windows 10
python 3.6.7
jupyter 1.0.0

improve windows permissions error handling

• fix color "bleed" after permissions error
• possibly silence post-permissions error stack traces?

Improvements to getting started

• link to getting started needs to be more prominent in the readme
  -test notebooks should be platform-neutral
  -in general, getting_started needs to be reworded to be clearer and have more of a walkthrough flow
  -elaborate no parameter viewing in getting_started
  -add information about creating rulesets in getting_started
  -test_ruleset should maybe be related to test_notebooks? for the walkthrough

right after the installation of praxxis, "prax al" won't be enough to load libraries

I noticed that when praxis is used for the very first time, running "prax al", it won't load properly the notebooks. Instead, it will appear an error like the one in the image.
The trick to praxxis really begin to recognize the notebooks is to run "prax l" immediately after "prax al". So "prax l" won't serve only to show the notebooks available :/

Sorry to bother you again :( and nice commits!

Allow deletion of rules by ordinal after prax vr

An idea that was floated, might involve adding some database stuff

add custom editor support

there should be a prompt on opening a notebook for the first time to pick which editor to use.

security questions

Right now, it's easy to see that even sensitive data is being stored in an insecure way though sqlite. We haven't really come to a conclusion on how we're going to fix something like that. It's more secure than typing your password into a shareable notebook, but it's just about as secure as saving your passwords in a file on your machine.

we were thinking of adding a -s flag when setting a parameter, and encrypting the db, but that would also make it difficult to share scenes.

We were thinking of encrypting with gpg, to allow people to share public keys and access encrypted sqlite dbs.