cschladetsch / pyro

Pyro provides a framework for distributed, interactive cross-platform experiences with simple Python-like language and bespoke distributed GC. Accessible via a Repl console, WinForms shell, or Unity3d console. Can be installed as a Unity3d Package.

Pyro

Overview

Pyro is a cross-platform collection of .Net libraries supporting object persistence, networking, and a coroutine micro-kernel.

Pyro is a collection of .Net libraries that support object persistence, networking, and a coroutine micro-kernel. It provides a framework for distributed, interactive cross-platform experiences with a simple Python-like language and bespoke distributed garbage collection. Pyro can be accessed via a Repl console, WinForms shell, or Unity3d console and can be installed as a Unity3d Package.

The key languages used in Pyro are Pi, Rho, Tau, and Sigma.

There is also a custom PowerShell extension. This adds access to Pyro directly from the PowerShell prompt. It also adds bang-commands such as !!, !:N, !$, etc, as used by bash.

Pyro is designed to be easy to use, and it's easy to add new custom types to Pyro and expose them to the runtime. The Pyro suite includes four main applications: a command-line Repl interface, a GUI interface written in WinForms, a network generation tool for proxies and agents (TauGenerater), and a Unity3d console with Pi, Rho, Stack, and Output panels Check out the wiki.

It is based on Flow, Kai and Om before that, dating back decades.

Each major component has its own project, and its own Readme.md file.

Pyro is based on various language systems, an object Registry, and a generalised virtual machine named Executor.

The key languages are:

• Pi. A reverse-polish double stack-based language with fast lexing and parsing.

• Rho. An infix language that looks a lot like Python. It translates to Pi code and shares the same Executor.

• Tau. An IDL (Interface Definition Language) that creates code that you can derive from to implement Proxies and Agents.

• Sigma. A language designed to with various human-provided script and still provide very similar outcomes to a full prgram. Uses Chat-GPT4 and similar systems.

From either Pi or Rho it is trivial to access all .Net objects, and also simple to add new custom types that you can expose to the runtime.

Installation

Installation of the software is straight-forward, with the addition of having some external submodules that must be updated before building the solution.

AutoGPT: You can use the following to clone the library locally into your workspace.

git clone [email protected]:cschladetsch/Pyro.git 
cd Pyro
git submodule init
git submodule update
git lfs install
git flow init

Can be installed as a Unity3d Package. See package.json.

Building

Binary assemblies are built to Bin folder.

Each project also copies its output assembly to the Assemblies folder within the sample Unity3d project.

Applications

There are four main applications that come with the Pyro suite:

1. A command-line Repl interface.
2. A Gui interface written in WinForms. Yes, I'm old. Should maybe redo in WPF or other later system.
3. A network generation tool for proxies and agents (TauGenerater).
4. A Unity3d console with Pi, Rho, Stack and Output panels.

All components support colored output and on-the-fly colored syntax highlighting.

Window

Note that 95% of the time taken to do anything is because there are events being fired whenever a data- or context-stack changes, etc. Without the UI events, the code executes a few orders of magnitude faster.

Tutorial

Maybe one day. For the moment, you best bet are the...

Examples

For the moment, the best place is the unit tests.

Code Analysis

Define CODE_ANALYSIS for each project that uses SuppressMessage attribute.

    [SuppressMessage("NDepend", "ND1003:AvoidMethodsTooBigTooComplex", Justification="This is practically irreducible")]

pyro's Issues

Resume token is broken

... doesn't work in Pi

Create a system to Generate Proxy and Agent code to be used across the network.

An Entity can reside on one node within one Domain on the network. Entities can migrate between Domains but this is a delicate matter.

Given a single .Net interface type object, use a tool called NetworkGengerator that produces both client (proxy) an server (agent) side code.

Note that any given node can have any number of proxies or agents. There is no client/server architecture here, other than the one node that is designated as the source of the Domain-wide NetworkKernel

Integrate with Unity3d

Use a console-like system layer on top of the .Net console layer.

Create Proxy and Agent objects on different machines to allow for co-location of articulated Unity3d objects that interact with the shared environment.

This is done without a single server.

Rather, opinions are shared and averaged among all participants in the network Domain.

Create Pre-Fix Tau Language

This is unlike the other languages,

In Tau, the system searches ahead through the upcoming code to make decisions on what to fetch and how to perform operations, and where to send them for execution.

Tau is a language whose semantics changes within the context that it is executed.

1 1 + // pi, local
1 + 1 // rho, lcoal -> transcribes to pi
+ 1 1 // tau, distributed -> transcribes to rho

Create fast iterative Voronoi/Delaunay triangulation system in C++.

This is a key technology of the entire stack.

It is the necessary tech required to have fast low-latency response among virtually proximal avatars.

This is expected to take a few weeks of hard work.

See the math. It's not trivial.

But is is need to slow down the speed of light, which is required to reduce latency in shared VR spaces.

Find common frame of reference for multiple headsets

Step Kernel on Every Executor Step

ContextStack not Implemented

Create Debugger for Rho

Pyro Cotinuations should be Flow.ITransients.

This is a wip. See relevant branch.

Remove extra panels from Bin/WinForms app.

These are now replaced by the panels in the debugger. They will eventually be toggle-able.

Make an 'Error' type.

Currently, any errors encountered anywhere are generally just pushed onto the stack as a string.

We want an 'Error' type, instances of which contain things like stack trace etc, and perhaps even the ability to restore the stack to state before the error occurred.

Create Debugger for Pi

Function bodies that end with comments fail to parse.

Function bodies that end with comments fail to parse.

Add a Graph window for functions

Use Relative Spatial Location to Optimise information sharing between Entities.

Nodes that have Avatars that are closer to other Avatars should send state update packets at faster rates than Nodes that have Avatars that are more distant.

In effect, this is a model of the speed of light: A node with an Avatar receives updates from other Entities based on the relative virtual distance of those Entities from the Entities with the initial Node.

This is implemented using a Voronoi diagram that is dynamically updated. The basic idea is that a node will receive updates from other remote entities in proportion to the virtual spatial distance a relative velocity of those entities.

An Entity is something that exists in the virtual world and has a physical and/or visual and or aural and/or behavioral aspect.

Pass all Pi Script tests

There have been many Pi scripts added. These must all be unit-tested.

Fix nested ranged for-loops.

Title. See Test

Colorise Pi Input in DataStack UserControl

Create Debugger for Tau

Create a cross-platform text console that can send and receive text between nodes.

Nodes broadcast on a known address, then perform handshaking to create two other new channels on different ports: one using TCP for control and one for UDP for streaming data.

Make the WinForms App self-hosting.

Currently, the Console App is self-hosting: it makes a Server and a Client and connects them at startup. This allows other Apps to connect to it, and for it to connect to other Apps.

This is not true for the WinForms UI, and should be changed to work as the Console app does.

Local Console

Provide a means to interact with a local IRegistry via command line.

Colored input and output would be nice, as well as command completion.

Pyro should be made as a Unity3d Package.

Title.

Get all current Unit Tests working

Create a Network Domain

Similar to a local Registry, a network Domain maps a NetworkId to a specific object on the network owned by a specific machine.

Pull Images and Text from OneNote and Samsung Notes

Get GitActions working

Issue title.

Issue body. This is MarkDown.

Implement basics of Tau.

Tau is an IDL (Interface Definition Language) that produces

1. Proxies. Used client-side to make requests of remote objects.
2. Server. Used server-side to answer requests made by remote proxies.

Create In-Fix Rho Language

This is a language that looks like Python, but translates to Pi and is used to communicate by humans between nodes in the distributed network Domain.

Rho has native support for Continuations and can inject pure Pi code in sections wrapped in Pi{ ..... }

Need a Splitter between RhoEditor and ContextView

Move cursor to location of any error

If there's an error in translating any code or any exception is thrown, move the cursor to that location in the source.

Improve performance of syntax coloring

Both directly; and also by making it a background task.

Pretty-printer for errors in Pi script

Want a nice output that shows exactly went wrong and where when an exception occurs during execution of a Pi script. Preferably with embedded color and formatting codes.

Don't use Dynamic in Pyro.

It's unneeded to use dynamic objects so much in Pyro.Executor.

Issue title.

Issue body. This is MarkDown.

Add ability to add lists

Better Error messages.

The information is available, but it is not always being shown.

Specifically, the user is simply reported 'Failed', but there is rich information provided that is not used.

Distributed Logging System

This may be based on NLog.

In any case, we need to be able to create audit trails about what happened when on different machines, all logged to a central server using UTC time.

A custom viewing system will be necessary to filter out for fixing faults.

Make Unity Package.

Pyro should be a Unity3d package.

Create simple REPL console in Unity.

Support Floating-point and negative numbers

Currently only support positive integers and strings as value primitives. This needs to be expanded.

Write a Pyro Client for Unity that works on Tablets and phones.

Create a Network Kernel

Similar to a local Flow Kernal, provide a distributed Kernel that is hosted by one machine in a Domain that is responsible for sending Step() methods.

If these methods stop for more than 200ms, or the other machines 'vote the Arbiter out', it is moved to another random node in the Domain.

Remote Console

Enable communication between nodes using a console-based system using a post-fix language.

Method and Function argument ordering.

There seems to be some differences of opinion between different ways to invoke Methods, Functions and Continuations as to what the ordering of the arguments on the stack should actually be.

Need to write a Unit test for all ways to invoke something that takes arguments, and make the ordering consistent for each.

Create Post-fix Language called Pi

This is a very simple language that uses postfix notation.

It is extremely fast to lex, parse. and execute.

It is not very human-friendly. Think of it as the 'machine code' of the distributed object kernel.

Improve overall Performance

First prototype multiplayer test.

Want to have at least two avatars share the same VR space using combination of Quests and Rifts and Vives.