breuleux / jurigged

Hot reloading for Python

jurigged

Jurigged lets you update your code while it runs. Using it is trivial:

1. jurigged your_script.py
2. Change some function or method with your favorite editor and save the file
3. Jurigged will hot patch the new function into the running script

Jurigged updates live code smartly: changing a function or method will fudge code pointers so that all existing instances are simultaneously modified to implement the new behavior. When modifying a module, only changed lines will be re-run.

You can also optionally install the develoop, a terminal-based live development environment:

As seen above, jurigged --loop <function_name> script.py will "loop" on a particular function of the script. That funtion will be re-run every time the source code is modified, with changes hot-patched into the running process. The rest of the program is not re-run, so preprocessing is preserved and heavy modules do not have to be reloaded!

Install

Jurigged requires Python version >= 3.8.

pip install jurigged

To also install the develoop feature, which lets you interactively develop functions:

pip install jurigged[develoop]

Command line

The simplest way to use jurigged is to add -m jurigged to your script invocation, or to use jurigged instead of python. You can use -v to get feedback about what files are watched and what happens when you change a file.

python -m jurigged -v script.py

OR

jurigged -v script.py

With no arguments given, it will start a live REPL:

python -m jurigged

OR

jurigged

Full help:

usage: jurigged [-h] [--interactive] [--watch PATH] [--debounce DEBOUNCE] [--poll POLL] [-m MODULE] [--dev] [--verbose] [--version]
                [SCRIPT] ...

Run a Python script so that it is live-editable.

positional arguments:
  SCRIPT                Path to the script to run
  ...                   Script arguments

optional arguments:
  -h, --help            show this help message and exit
  --interactive, -i     Run an interactive session after the program ends
  --watch PATH, -w PATH
                        Wildcard path/directory for which files to watch
  --debounce DEBOUNCE, -d DEBOUNCE
                        Interval to wait for to refresh a modified file, in seconds
  --poll POLL           Poll for changes using the given interval
  -m MODULE             Module or module:function to run
  --dev                 Inject jurigged.loop.__ in builtins
  --verbose, -v         Show watched files and changes as they happen
  --version             Print version

Develoop

Usage:

# Loop over a function
jurigged --loop function_name script.py
jurigged --loop module_name:function_name script.py

# Only stop on exceptions
jurigged --xloop function_name script.py

The "develoop" is an optional feature of Jurigged that provides a sort of live development environment for a function. If you run jurigged --loop <function_name> <script>, the function of that name in the script will be part of the "develoop". When it is entered, it will be run, its output will be captured and displayed, and the program will wait for input. If the source code is changed, the function will run again.

The --xloop or -x flag works the same, but the loop is only done if the function raises an exception. If it does not raise an exception, it will run like normal. Both --loop and --xloop can be used multiple times, if you want to loop over multiple functions.

The default interface allows a few commands:

• r to manually re-run the loop. This can be done in the middle of a run.
• a to abort the current run (e.g. if you get stuck in an infinite loop).
• c to exit the loop and continue the program normally.
• q to quit the program altogether.

Using with stdin

The default develoop interface does not play well with stdin. If you want to read from stdin or set a breakpoint(), use the decorator @__.loop(interface="basic"). The interface will be cruder, but stdin/pdb will work.

Troubleshooting

First, if there's a problem, use the verbose flag (jurigged -v) to get more information. It will output a Watch <file> statement for every file that it watches and Update/Add/Delete <function> statements when you update, add or delete a function in the original file and then save it.

The file is not being watched.

By default, scripts are watched in the current working directory. Try jurigged -w <file> to watch a specific file, or jurigged -w / to watch all files.

The file is watched, but nothing happens when I change the function.

You can try using the --poll <INTERVAL> flag to use polling instead of the OS's native mechanisms. If that doesn't work, try and see if it works with a different editor: it might have to do with the way the editor saves. For example, some editors such as vi save into a temporary swap file and moves it into place, which used to cause issues (this should be fixed starting with v0.3.5).

Jurigged said it updated the function but it's still running the old code.

If you are editing the body of a for loop inside a function that's currently running, the changes will only be in effect the next time that function is called. A workaround is to extract the body of the for loop into its own helper function, which you can then edit. Alternatively, you can use reloading alongside Jurigged.

Similarly, updating a generator or async function will not change the behavior of generators or async functions that are already running.

I can update some functions but not others.

There may be issues updating some functions when they are decorated or stashed in some data structure that Jurigged does not understand. Jurigged does have to find them to update them, unfortunately.

API

You can call jurigged.watch() to programmatically start watching for changes. This should also work within IPython or Jupyter as an alternative to the %autoreload magic.

import jurigged
jurigged.watch()

By default all files in the current directory will be watched, but you can use jurigged.watch("script.py") to only watch a single file, or jurigged.watch("/") to watch all modules.

Recoders

Functions can be programmatically changed using a Recoder. Make one with jurigged.make_recoder. This can be used to implement hot patching or mocking. The changes can also be written back to the filesystem.

from jurigged import make_recoder

def f(x):
    return x * x

assert f(2) == 4

# Change the behavior of the function, but not in the original file
recoder = make_recoder(f)
recoder.patch("def f(x): return x * x * x")
assert f(2) == 8

# Revert changes
recoder.revert()
assert f(2) == 4

# OR: write the patch to the original file itself
recoder.commit()

revert will only revert up to the last commit, or to the original contents if there was no commit.

A recoder also allows you to add imports, helper functions and the like to a patch, but you have to use recoder.patch_module(...) in that case.

Caveats

Jurigged works in a surprisingly large number of situations, but there are several cases where it won't work, or where problems may arise:

• Functions that are already running will keep running with the existing code. Only the next invocations will use the new code.
  • When debugging with a breakpoint, functions currently on the stack can't be changed.
  • A running generator or async function won't change.
  • You can use reloading in addition to Jurigged if you want to be able to modify a running for loop.
• Changing initializers or attribute names may cause errors on existing instances.
  • Jurigged modifies all existing instances of a class, but it will not re-run __init__ or rename attributes on existing instances, so you can easily end up with broken objects (new methods, but old data).
• Updating the code of a decorator or a closure may or may not work. Jurigged will do its best, but it is possible that some closures will be updated but not others.
• Decorators that look at/tweak function code will probably not update properly.
  • Wrappers that try to compile/JIT Python code won't know about jurigged and won't be able to redo their work for the new code.
  • They can be made to work using a __conform__ method (see below).

Customizing behavior

In order to update a transform of a Python function, for example a transform that generates a new code object based on the original source code, you need to do something like this:

class Custom:
    __slots__ = ("code",)

    def __init__(self, transformed_fn, code):
        self.code = code
        self.transformed_fn = transformed_fn

    def __conform__(self, new_code):
        if new_code is None:
            # Function is being deleted
            ...

        if isinstance(new_code, types.FunctionType):
            new_code = new_code.__code__

        do_something(new_code)
        self.code = new_code

...
transformed_fn.somefield = Custom(transformed_fn, orig_fn.__code__)

Basically, when the original code is changed, jurigged will use the gc module to find objects that point to it, and if they have a __conform__ method it will be called with the new code so that the transformed function can be updated. The original code must be in a slot on that object (it is important that it is in __slots__, otherwise the referrer is a dictionary). Multiple transformed functions may exist.

How it works

In a nutshell, jurigged works as follows:

1. Inventory existing modules and functions: a. Insert an import hook that collects and watches source files. b. Look at all existing functions using gc.get_objects(). c. Add an audit hook that watches calls to exec in order to inventory any new functions.
2. Parse source files into sets of definitions.
3. When a file is modified, re-parse it into a set of definitions and match them against the original, yielding a set of changes, additions and deletions.
4. When a function's code has changed: a. Strip out the decorators b. Execute the new code c. Use gc.get_referrers() to find all functions that use the old code d. Replace their internal __code__ pointers
5. If the replacement fails or if brand new code is added, execute the new code in the module namespace.

Comparison

The two most comparable implementations of Jurigged's feature set that I could find (but it can be a bit difficult to find everything comparable) are %autoreload in IPython and limeade. Here are the key differences:

• They both re-execute the entire module when its code is changed. Jurigged, by contrast, surgically extracts changed functions from the parse tree and only replaces these. It only executes new or changed statements in a module.

  Which is better is somewhat situation-dependent: on one hand, re-executing the module will pick up more changes. On the other hand, it will reinitialize module variables and state, so certain things might break. Jurigged's approach is more conservative and will only pick up on modified functions, but it will not touch anything else, so I believe it is less likely to have unintended side effects. It also tells you what it is doing :)

• They will re-execute decorators, whereas Jurigged will instead dig into them and update the functions it finds inside.

  Again, there's no objectively superior approach. %autoreload will properly re-execute changed decorators, but these decorators will return new objects, so if a module imports an already decorated function, it won't update to the new version. If you only modify the function's code and not the decorators, however, Jurigged will usually be able to change it inside the decorator, so all the old instances will use the new behavior.

• They rely on synchronization points, whereas Jurigged can be run in its own thread.

  This is a double-edged sword, because even though Jurigged can add live updates to existing scripts with zero lines of additional code, it is not thread safe at all (code could be executed in the middle of an update, which is possibly an inconsistent state).

Other similar efforts:

• reloading can wrap an iterator to make modifiable for loops. Jurigged cannot do that, but you can use both packages at the same time.

jurigged's Issues

File change not detected

I have the following snippet:

import time

import jurigged


def experiment(*args, **kwargs):
    def decorator(func):

        while True:
            time.sleep(0.1)
            res = func(*args, **kwargs)
            print(repr(res), end="\r")

    return decorator


if __name__ == "__main__":

    @experiment()
    def main():
        a = 2
        return 1

If I run:

jurigged -v essai.py 
Watch /tmp/essai.py

I do get "1", but if I modify the code, jurigged doesn't "Update main.main" as usual. I suppose the decorator does something to it.

ARM64 issue

File "/root/PycharmProjects/pythonProject/.venv/lib/python3.10/site-packages/watchdog/observers/inotify_c.py", line 415, in _raise_error
raise OSError(errno.EMFILE, "inotify instance limit reached")
OSError: [Errno 24] inotify instance limit reached

Steps to replicate :
jurigged -v filename.py

OS : ubuntu 64 - running in arm64

Can't pass arguments that start with -- to module

If you attempt to do jurigged -m module --argument 123, argparse inside jurigged will get angry and throw unrecognized arguments.

As a workaround you can pass "" after -m module.

Workaround for Geany IDE: Detect other types of file modifications: file move, file close, file overwritten

This is related to how Geany handles files by default: It creates a temporary file copy, writes the modifications to it and them copies the modifications to the current file.
These steps are made to prevent file corruption when there is no space left in the device.
Unfortunately this is not detected as a modification by jurigged and the file is not hot-reloaded

these are the events I logged by using on_any_event inside the class JuriggedHandler(FileSystemEventHandler):

event type: created path src : app_path/test_flask_hotreloading/.goutputstream-6PJ0X1
event type: modified path src : app_path/test_flask_hotreloading
event type: modified path src : app_path/test_flask_hotreloading/.goutputstream-6PJ0X1
event type: closed path src : app_path/test_flask_hotreloading/test_flask_hello.py
event type: modified path src : app_path/test_flask_hotreloading
event type: modified path src : app_path/test_flask_hotreloading/.goutputstream-6PJ0X1
event type: moved path src : app_path/test_flask_hotreloading/.goutputstream-6PJ0X1
event type: modified path src : app_path/test_flask_hotreloading
event type: closed path src : app_path/test_flask_hotreloading/test_flask_hello.py
event type: modified path src : app_path/test_flask_hotreloading

I propose the following temporary workaround:

    on_closed = on_modified

however, the best way can be to use only an on_any_event event and them, inside it handle all possible situations:

def on_any_event(self, event):
    print(f'event type: {event.event_type}  path src : {event.src_path}', flush = True)

a more aggressive approach:

def on_any_event(self, event):
    print(f'event type: {event.event_type}  path src : {event.src_path}', flush = True)
    if ".py" in event.src_path:
        #do reload procedure

I've not made a pull request because it is open to discussion, but I'm using the temporary workaround.

AttributeError: 'NoneType' object has no attribute '__code__'

When running the following self-contained example I'm hit with an AttributeError

Traceback (most recent call last):
  File "/home/user/env/lib/python3.11/site-packages/jurigged/codetools.py", line 493, in _process_child_correspondence
    orig.apply_correspondence(
  File "/home/user/env/lib/python3.11/site-packages/jurigged/codetools.py", line 704, in apply_correspondence
    new_obj = self.reevaluate(corr.new.node, glb)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/env/lib/python3.11/site-packages/jurigged/codetools.py", line 801, in reevaluate
    conform(self.get_object(), new_obj)
  File "/home/user/env/lib/python3.11/site-packages/codefind/registry.py", line 118, in conform
    self.conform(obj1, obj2.__code__)
  File "/home/user/env/lib/python3.11/site-packages/codefind/registry.py", line 123, in conform
    fv1 = obj1.__code__.co_freevars
          ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '__code__'

Any idea of what could be going wrong?

import asyncio
from pathlib import Path

import jurigged


async def slave():
    try:
        print('hello there')
        print("starting A")
        while True:
            print("A")
            await asyncio.sleep(1)
    except asyncio.CancelledError:
        print("stopped A")


async def master(tg: asyncio.TaskGroup):
    while True:
        task = asyncio.create_task(slave())
        await asyncio.sleep(4)
        task.cancel()


async def main():
    async with asyncio.TaskGroup() as tg:
        tg.create_task(master(tg))


if __name__ == "__main__":
    jurigged.watch(str(Path.cwd() / "asynctgtest.py"))
    asyncio.run(main())

Pytest hot reloader issues

First off, thanks for this great library. I've used it to implement a pytest hot reloading daemon. I had some issues that I worked around using monkey patching, and I think it would make sense to update the library rather than use monkey patches.

Here's the list of issues I needed to work around:

• glob - needed more control over directories. You may have so many files, attempting to watch them all would exhaust the open file handle limit, so the ability to specify multiple root directories was needed.
• assertion rewrites - pytest does assertion rewrites. When jurigged would hot reload a test function, the test function would no longer have the assertions rewritten. I think the best solution to this would be to add hook capability on reevaluate
• line numbers off by one - I encountered this in two places. The first was rewritten functions having their line numbers shifted down by one line. The second issue is where there is an ensure_separation parameter in GroupDefinition.append that inserts a blank line. This creates line breaks where they don't exist, resulting in line numbers diverging after certain things in the file. It's not clear to me the purpose of this. When line numbers are off, the debugger and introspection libraries like varname quit working as expected, on top of errors giving the wrong line number.
• jurigged logger - the default behavior was resulting in duplicated output
• watchman - this isn't really on jurigged specifically but watchman seems to have compatibility problems so I use polling. I've been wanting to create a library that does file watching better by doing a hybrid of signals and polling but haven't gotten around to it.

Here's the full code of the monkey patches:

def monkey_patch_jurigged_function_definition():
    import jurigged.codetools as jurigged_codetools  # type: ignore
    import jurigged.utils as jurigged_utils  # type: ignore

    OrigFunctionDefinition = jurigged_codetools.FunctionDefinition

    import ast

    class NewFunctionDefinition(OrigFunctionDefinition):
        def reevaluate(self, new_node, glb):
            # monkeypatch: The assertion rewrite is from pytest. Jurigged doesn't
            #              seem to have a way to add rewrite hooks
            new_node = self.apply_assertion_rewrite(new_node, glb)
            obj = super().reevaluate(new_node, glb)
            return obj

        def apply_assertion_rewrite(self, ast_func, glb):
            from _pytest.assertion.rewrite import AssertionRewriter

            nodes: list[ast.AST] = [ast_func]  # type: ignore
            while nodes:
                node = nodes.pop()
                for name, field in ast.iter_fields(node):
                    if isinstance(field, list):
                        new: list[ast.AST] = []  # type: ignore
                        for i, child in enumerate(field):
                            if isinstance(child, ast.Assert):
                                # Transform assert.
                                new.extend(
                                    AssertionRewriter(glb["__file__"], None, None).visit(child)
                                )
                            else:
                                new.append(child)
                                if isinstance(child, ast.AST):
                                    nodes.append(child)
                        setattr(node, name, new)
                    elif (
                        isinstance(field, ast.AST)
                        # Don't recurse into expressions as they can't contain
                        # asserts.
                        and not isinstance(field, ast.expr)
                    ):
                        nodes.append(field)
            return ast_func

        def stash(self, lineno=1, col_offset=0):
            # monkeypatch: There's an off-by-one bug coming from somewhere in jurigged.
            #              This affects replaced functions. When line numbers are wrong
            #              the debugger and inspection logic doesn't work as expected.
            if not isinstance(self.parent, OrigFunctionDefinition):
                co = self.get_object()
                if co and (delta := lineno - co.co_firstlineno):
                    delta -= 1  # fix off-by-one
                    if delta != 0:
                        self.recode(jurigged_utils.shift_lineno(co, delta), use_cache=False)

            return super(OrigFunctionDefinition, self).stash(lineno, col_offset)

    # monkey patch in new definition
    jurigged_codetools.FunctionDefinition = NewFunctionDefinition


def monkeypatch_group_definition():
    import jurigged.codetools as jurigged_codetools  # type: ignore

    def append(self, *children, ensure_separation=False):
        for child in children:
            # ensure_separation creates line number diff
            # an example where this was a problem:
            #
            # 15 class MyClass:
            # 77     do_something()  # type: ignore  <--- blank line inserted between do_something() and comment
            # 78
            # 79     def my_func(...)  <--- becomes line 80
            #
            # the monkey patch removes it
            #
            # removed code:
            # if (
            #     ensure_separation
            #     and self.children
            #     and not self.children[-1].well_separated(child)
            # ):
            #     ws = LineDefinition(
            #         node=None, text="\n", filename=self.filename
            #     )
            #     self.children.append(ws)
            #     ws.set_parent(self)
            self.children.append(child)
            child.set_parent(self)

    jurigged_codetools.GroupDefinition.append = append


def setup_jurigged(config: Config):
    def _jurigged_logger(x: str) -> None:
        """
        Jurigged behavior is to both print and log.

        By default this creates duplicated output.

        Pass in a no-op logger to prevent this.
        """

    import jurigged

    monkey_patch_jurigged_function_definition()
    monkeypatch_group_definition()

    pattern = _get_pattern_filters(config)
    # TODO: intelligently use poll versus watchman (https://github.com/JamesHutchison/pytest-hot-reloading/issues/16)
    jurigged.watch(pattern=pattern, logger=_jurigged_logger, poll=True)

jurigged -v <script.py> executes and exits

hi.py:

 print('hi')                                              

jurigged -v hi.py      
Watch /path/hi.py
hi

The process simply exits after executing the script once. How do I get the reloading to work on a M1 mac?

Q: Watch entire module, updated via git, with possibly new files added to the module

I'm trying to implement a small application that's going to hot reload its code from a remote repository. What it does is simple: every 5 minutes, it fetches the remote branch origin/main and updates the mymodule code to take the new version of the code into account in the running process.

Q1. How can I make jurigged take into account potentially new files once they're added to my remote git repository and fetched locally?

Q2. Is jurigged going to behave nicely if this runs in an asyncio loop? With a coroutine that fetches the latest git version every 5 minutes?

Thanks!

import pathlib

import git
import jurigged
import mymodule
from loguru import logger

MYMODULE_DIR_PATH = pathlib.Path(mymodule.__path__[0]).parent
if not (MYMODULE_DIR_PATH / ".git").is_dir():
    raise RuntimeError(f"could not find git repo in {MYMODULE_DIR_PATH}")
jurigged.watch(str(MYMODULE_DIR_PATH.absolute()))
MYMODULE_REPO = git.Repo(MYMODULE_DIR_PATH)


def update_code_from_git(branch: str = "main") -> None:
    cur_hash = MYMODULE_REPO.remotes.origin.refs[branch].commit.hexsha
    logger.info(
        f"fetching origin/{branch} for repo mymodule. current hash is {cur_hash}"
    )
    MYMODULE_REPO.remotes.origin.fetch(branch)
    new_hash = MYMODULE_REPO.remotes.origin.refs[branch].commit.hexsha
    if new_hash != cur_hash:
        logger.info(
            f"new hash {new_hash} found. code will be hard reset to origin/{branch}"
            " and hot reloaded"
        )
        MYMODULE_REPO.heads[branch].reset(
            MYMODULE_REPO.remotes.origin.refs[branch], hard=True
        )
    else:
        logger.info("hash did not change")


def main():
    # TODO: implement
    pass


if __name__ == "__main__":
    main()

Problem with non-atomic writes

This problem reported in a comment on Hacker News.

I think some editors might empty files before writing to them, which leads Jurigged to behave sub-optimally and/or break when changing the top level module (it will "delete" all statements and then re-execute them, including the starting loop). Whether or not that's the actual issue behind the problem described here, it would be a good idea to debounce modifications so that non-atomic writes work if they operate fast enough.

Maybe also add a --delay flag on the CLI to control that, which people might find useful in other situations.

Idea for mapping between codes and functions

Here's an idea for how you might be able to find the correct code and function objects. I'm not 100% sure if this would actually fit in the way you're doing things but I thought it might help. I also don't actually know how reliable gc is for this, it might be, but I haven't really investigated.

import gc
import types


def find_subcodes(code):
    """
    Yields all code objects descended from this code object.
    e.g. given a module code object, will yield all codes defined in that module.
    """
    for const in code.co_consts:
        if isinstance(const, types.CodeType):
            yield const
            yield from find_subcodes(const)


def get_functions(code):
    """
    Returns functions that use the given code.
    """
    return [
        ref
        for ref in gc.get_referrers(code)
        if isinstance(ref, types.FunctionType)
           and ref.__code__ == code
    ]


module_code = compile("""
def foo():
    def bar():
        pass
    return bar
""", "<string>", "exec")

codes = list(find_subcodes(module_code))

exec(module_code)

bars = [foo(), foo()]

code_to_functions = {code: set(get_functions(code)) for code in codes}

print(code_to_functions)

assert code_to_functions == {foo.__code__: {foo}, bars[0].__code__: set(bars)}

`from __future__ import annotations` not respected

on recompile of a function definition change, jurigged does not respect from __future__ import annotations

say you have a module like this:

from __future__ import annotations

def f(a: tuse) -> int:
    ...

and say you run jurigged my_module.py
and say you additionally change:

def f(a: tuse) -> int:
# to
def f(a: tuse) -> str:

jurigged raises an error in reevaluation (NameError: tuse is not defined or something like that).

the reason is that the appropriate future flags aren't passed into the compile function.

i'll push a fix for this shortly, just wanted to create an issue first.

Reload a specific module

Hi! I'm interested in a use case where I only want to reload a particular library that I am developing, but I could change any function or file within that library. Would one of the following snippets achieve this, inside of IPython for example?

import jurigged
jurigged.watch(mymodule.__file__)

import pathlib
import jurigged
mydir = str(pathlib.Path(mymodule.__file__).parent)
jurigged.watch(mydir)

And if so would it make sense to add jurigged.watch(mymodule) as syntactic sugar?

UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 594: illegal multibyte sequence

Traceback (most recent call last):
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\site-packages\jurigged\__main__.py", line 4, in <module>
    cli()
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\site-packages\jurigged\live.py", line 258, in cli
    mod, run = find_runner(opts, pattern)
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\site-packages\jurigged\live.py", line 183, in find_runner
    registry.prepare("__main__", path)
  File "C:\Users\InPRTx\AppData\Local\Programs\Python\Python38\lib\site-packages\jurigged\register.py", line 60, in prepare
    f.read(),
UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 594: illegal multibyte sequence

jurigged 0.3.3

It seems that it is not set to open the file with utf8 encoding by default

jurigged/jurigged/register.py

Line 57 in 75dc1a5

with open(filename) as f:

watchdog version is incompatible

When I install jurigged, by default it requires watchdog version <2.0.0, >=1.0.2 and it automatically install watch 1.0.2. But mkdocs 1.2.2 is also a dependency of jurigged which requires watchdog >=2.0. The wathcdog 1.0.2 which install by default is incompatible with it. How to solve this dependency issue?

Tkinter applications

Does this work with Tkinter applications or will those break?

Files not reloaded with PyCharm with Windows

Description

On Windows, jurigged does not seem to reload a file it's running after I edit it with PyCharm. I have read this may be an issue with how WatchDog monitors for file updates, but I didn't see JetBrains IDEs mentioned specifically so I'm not sure if it's expected that this would work on them.

It may be nice to include a table of editors which are known to be compatible with jurigged, and any notes for making them work if necessary.

Steps to reproduce

1. Create a new project in PyCharm
2. Write the script

from time import sleep

def do_something():
    print("Hello")

def main():
    for _ in range(100):
        do_something()
        sleep(1)

if __name__ == '__main__':
    main()

3. Run the script with jurigged -v main.py in your terminal
4. As the script is printing "Hello" repeatedly, change print("Hello") to print("Hi") and click out of the file so it saves.

Expected behavior

The script would go from printing "Hello" to "Hi"

Actual behavior

The script continues to print "Hello"

Platform

PyCharm version

PyCharm 2021.2.2 (Professional Edition)
Build #PY-212.5284.44, built on September 14, 2021

Python version

Python 3.10.0

Dependency versions

ansicon==1.89.0
blessed==1.19.0
codefind==0.1.2
jinxed==1.1.0
jurigged==0.3.4
ovld==0.3.2
six==1.16.0
watchdog==1.0.2
wcwidth==0.2.5

Device specifications

Processor	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz   3.60 GHz
Installed RAM	32.0 GB (31.9 GB usable)
System type	64-bit operating system, x64-based processor

Windows specifications

Edition	Windows 11 Education Insider Preview
Version	Dev
Installed on	‎10/‎19/‎2021
OS build	22478.1012
Experience	Windows Feature Experience Pack 1000.22478.1012.0

Support for pre- and post-update hooks

Essentially, I was hoping that it were possible to add support for custom hooks to run before and after updating the code in memory so that you can run pre- and post-update tasks.

This is a fairly niche use case, but a Discord bot framework I'm working on (https://github.com/NAFTeam/NAFF) is looking to implement jurigged for fast-paced development of Discord bots, and re-registering command with Discord whenever files are changed. However, it's not really feasible, as doing so requires tracking everything both before and after the change, which currently isn't possible (that I'm aware of)

Errorr using jurigged on docker

I am getting an error when i am using with docker:

Traceback (most recent call last):
  File "/usr/local/bin/jurigged", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/jurigged/live.py", line 268, in cli
    run()
  File "/usr/local/lib/python3.9/site-packages/jurigged/live.py", line 188, in run
    runpy.run_path(path, module_object=mod)
  File "/usr/local/lib/python3.9/site-packages/jurigged/runpy.py", line 279, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "/usr/local/lib/python3.9/site-packages/jurigged/runpy.py", line 252, in _get_code_from_file
    with io.open_code(decoded_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/srv/www/s'

Dockerfile

FROM python:3.9-slim

RUN mkdir /srv/www/

ADD ./ /srv/www/

WORKDIR /srv/www/

ENV GRPC_PYTHON_VERSION 1.39.0
RUN python -m pip install --upgrade pip
RUN pip install grpcio==${GRPC_PYTHON_VERSION} grpcio-tools==${GRPC_PYTHON_VERSION} grpcio-reflection==${GRPC_PYTHON_VERSION} grpcio-health-checking==${GRPC_PYTHON_VERSION} grpcio-testing==${GRPC_PYTHON_VERSION}

RUN pip install -r requirements.txt

RUN pip install jurigged
ENTRYPOINT ["jurigged"]
CMD ["main.py"]

Python file main.py

import time
import datetime

def main():
    while True:
        print("Executed %s" % datetime.datetime.now())
        time.sleep(5)


main()

latest jurigged does not work with CPython 3.12.0

I have used jurigged successfully and smoothly with Python 3.11 since the first release of the latter. In the past week, I have tried upgrading to CPython 3.12.0 (on Linux) and while everything loads and runs as normal, hot-reloading doesn't happen. Reverting the .venv/ to CPython 3.11 makes it work as before, as expected.

I am sorry for the lack of details and debugging; I mainly wanted to open this item as a sanity check for others. :-)

No matching distribution found for jurigged

ERROR: Could not find a version that satisfies the requirement jurigged (from versions: none)
ERROR: No matching distribution found for jurigged

Pip 21.2.2
Python 3.7.11

Any idea on how to fix that, please?

[Windows] Non-Latin characters are shown incorrectly upon function reload

Hello! Thank you for the library! I'm having one issue which prevents me from using jurigged on regular basis. The issue is with non-Latin characters. But enough talk, check the code:

def make_greet_message(name):
    return f"Привет, {name}!"


def main():
    while True:
        name = input("Enter your name: ")
        print(make_greet_message(name))


if __name__ == '__main__':
    main()

On the line 3 the function returns greeting text with Russian word Привет (Hello) and everything works fine:

Enter your name: alex
Привет, alex!

However, if I change text to some other non-Latin text, like "Добрый день, {name}!" (Good day), the output looks differently:

Enter your name: alex
Добрый день, alex!

I cannot reproduce this issue on Manjaro Linux, however this happens on Windows 11 22H2 with Russian language as default. What should I change to fix this? Testing on integrated PowerShell in PyCharm 2023.3.3

Improve import time

Hi,

I am profiling my application startup time. In the profile result, jurigged requires around 300ms, where 200ms comes from importing blessed. Since I only run live.watch() and blessed is only used to print terminal colors, this 200ms seems too much.

May I suggest using more lightweight library to print terminal colors in the live module (e.g. colorama), with a wrapper class

import colorama

class Terminal:
    def __init__(self):
        if sys.platform == 'win32':
            colorama.just_fix_windows_console()

    def __getattr__(self, name):
        styles = []
        for s in name.split('_'):
            match s:
                case 'bold':
                    styles.append(colorama.Style.BRIGHT)
                case other:
                    styles.append(getattr(colorama.Fore, other.upper()))
        return lambda text: ''.join(styles) + text + colorama.Style.RESET_ALL

T = Terminal()

I have added this to live.py and the import time is reduced to around 100ms.

jurigged.watch

Hi

i have script.py

import os 
import sys 
import numpy as np 
import numpy.typing as npt 
import runpy 
import jurigged 
os.environ['PYTHONIOENCODING'] = 'utf-8' 
watch_directory = '/ComfyUI/custom_nodes/ComfyUI_MaraScott_Nodes/__init__.py'  
jurigged.watch(watch_directory) 
sys.argv = ['jurigged'] + sys.argv[1:] 
runpy.run_module('jurigged', run_name='__main__', alter_sys=True) 

and

.\python_embeded\python.exe script.py -m jurigged -v "ComfyUI\main.py" --listen 127.0.0.1 --port 8188

but it seems it do not care about the directory I want to watch and it watches everything.

Any idea how can I fix that ?

decorated dataclass class not being reloaded on definition change

Hello,

jurigged works very well for me in how I use it with the cmd2 module. However, I have noted that it doesn't seem to be picking up changes to the definition of a dataclass (decorated with @dataclasses.dataclass() )

I expect this to be a known limitation, given the way class decorators carry out their work. Am I right in thinking that changes to the definition of a dataclass will not be currently picked up by jurigged? I'm using CPython 3.11.2 and jurigged 0.5.5 (from jurigged.version.version)

Thank you once again for jurigged. It has transformed multiple projects from misery to (near) pleasure :-)

Working great on Ubuntu, but not on Windows

First of all - I love this code. Thank you so much for creating it. I'm using it to create a live-coding environment to make music with SuperCollider.

I had no problems running jurigged on Ubuntu, but I can't get it to work on Windows. I tried running it the same way: python3 -m jurigged -v livecoding.py but on Windows it never registers any change in the file when I save. I'm using Sublime on Windows to save. My python version is 3.9.7 and Windows version is 10. Is there a known issue with Windows?

jurigged -w starts python interactive mode

jurigged -w should start jurigged watching only the specified file.

But: jurigged -w script.py shows this:

If I try: jurigged -w script.py script.py the script starts but It watch for the entire current directory instead of just the specified file.

I'd just want to start jurigged and watch for only one file, am I doing it wrongly?

Maybe a file name not supported by win was used

PS C:\Users\InPRTx\Desktop\git> git clone https://github.com/breuleux/jurigged.git
Cloning into 'jurigged'...
remote: Enumerating objects: 878, done.
remote: Counting objects: 100% (878/878), done.
remote: Compressing objects: 100% (457/457), done.
Receiving objects: 94% (826/878)used 618 (delta 393), pack-reused 0
Receiving objects: 100% (878/878), 203.28 KiB | 1.83 MiB/s, done.
Resolving deltas: 100% (612/612), done.
error: invalid path 'tests/snippets/ballon:main.py'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Default parameter value from class scope not seen by jurigged on method reload

environment
image:docker.io/library/python:3.9.6
docker: Docker version 24.0.7, build afdd53b (native Linux, Kubuntu)
jurigged: 0.5.7

script.py:

import time

class A:
    CONST = 12

    def m(self, val=CONST):
        print(val)

a = A()
while True:
    a.m()
    time.sleep(1)

cmd: jurigged script.py, then try modifying the method.

output:

12
12
12
Traceback (most recent call last):
  File "/opt/venv/lib/python3.9/site-packages/jurigged/codetools.py", line 493, in _process_child_correspondence
    orig.apply_correspondence(
  File "/opt/venv/lib/python3.9/site-packages/jurigged/codetools.py", line 704, in apply_correspondence
    new_obj = self.reevaluate(corr.new.node, glb)
  File "/opt/venv/lib/python3.9/site-packages/jurigged/codetools.py", line 790, in reevaluate
    exec(code, glb, lcl)
  File "/app/script.py", line 7, in <adjust>
    def m(self, val=CONST):
NameError: name 'CONST' is not defined