kafonek / ipython_blocking

A Python library that offers a context manager to turn on cell execution capture/blocking in Jupyter notebooks/IPykernel. Useful when you need a "blocking widget"

Hi, I'm @kafonek (Matt Kafonek) 👋

My first career was as a domain expert in the intelligence community. I fell into it as a young enlisted USAF linguist, and kept working in the field for almost twenty years as a geolocation specialist and technical analyst with VaST. I began learning Python around 2010 to help automate some day-to-day tasks. As I shared what I learned with other analysts, using tools like Jupyter, I developed a passion for empowering domain experts. One part of that is providing "no-code" / "low-code" tools, but an underserved need is giving them the right opportunity to begin learning and using code themselves.

My second career as a software engineer started with Noteable, a managed Notebook platform to make data accessible to all. I learned countless valuable lessons developing and deploying features for a production application. Our backend stack was Python (FastAPI) and CockroachDB (async SQLAlchemy), with the Kernel compute orchestrated through a Kubernetes Operator and custom resource definitions. We home-rolled RBAC and real-time synchronization 😬. When ChatGPT announced the plugin store, and later custom GPTs, we were in a unique position to be first-to-market with a Notebook/coding plugin, which landed on the initial "popular" store. I have seen what happens when a startup catches lightning in a bottle, and it is freaking cool.

My current interests are twofold: learning Rust and becoming more comfortable applying practical AI. I am excited about Rust because its syntax, build tools, and ecosystem are approachable. It's also blazing fast and compact. I believe the future of performant Python is Rust. Deploying apps to the browser with web-assembly has a lot of promise, and Rust has a tremendous story there. As far as AI goes, the experience with Noteable was "magical" but we were effectively just interacting with OpenAI APIs, whether it was ChatGPT making calls to us or our app making calls to OpenAI. I'm looking for opportunities to utilize local models, work with multi-modal models, and learn about deploying and optimizing models in production.

Outside of work I spend my time solving programming puzzles, playing video games, rock climbing with my wife, and taking care of our two young boys.

ipython_blocking's Issues

Support reading other properties than value?

Now my problem is that I want to use it with from ipyfilechooser import FileChooser, which doesn't inherit from ValueWidget or have a value property or something:

/opt/conda/lib/python3.7/site-packages/ipython_blocking/ipython_magic.py in block(self, line)
     57             func = lambda: obj._has_been_clicked
     58         else:
---> 59             raise Exception('The positional argument to %block should be a ValueWidget, Button, or a function/method')
     60         return self.capture(func, args.timeout)
     61 

If there were some way to point at a property: %block myobject.property, and somehow introspect that it was indeed a value or property to do change detection on, then it could be a good general-purpose solution. Also, FileChooser should probably actually inherit and provide a value, but that's another matter.

Yes, I know I can write a change-detecting function, as per the workaround in #5:

# previous cell
config_selector = FileChooser(str(root_working_dir), use_dir_icons=True)
display(config_selector)

# next cell
ctx = ipython_blocking.CaptureExecution()
with ctx:
    init_val = config_selector.selected
    while True:
        if init_val != config_selector.selected:
            break
        ctx.step()

...but I was hoping for an easy-to-read magic one-liner.

%blockrun button executes even after clicking the button on google colab

Error reproducible at
cell 1
try:
import ipython_blocking
except ImportError:
!pip install ipython_blocking
import ipython_blocking

cell 2
from ipywidgets import Dropdown, Button, Output
from IPython.display import display, clear_output

outs = Output()
query_selector = Dropdown(
options=['Select', 'Q1', 'Q2'],
value='Select',
description='Query Type:'
)

process_button = Button(
description='Process',
disabled=False
)

def proc_button(bb):
with outs:
clear_output()
display(process_button)

display(query_selector)
query_selector.observe(proc_button, names='value')
display(outs)

cell 3
%blockrun process_button

cell 4
print(query_selector.value)

causing the cell 3 to run continuously even after the button is selected.

Using "Run All" with ipython_blocking leads to unexpected execution order.

Using Jupyter's Run all can lead to unexpected execution order if one of the cells makes use of ipython_blocking. See the following minimal example:

Great job!

Hi Mat,

This is not so much an issue as it is an encouragement on your work with ipython_blocking. Its exactly what I was looking for, for hours today.

I will definitely send feedback if I have any and submit PR's if I can.

Thanks again!

Sohail

how to block python code execution till js snippet returns in Jupyter notebook

I am trying to execute a JS code to get the notebook name in Jupyter

import IPython
from IPython.core.magics.display import Javascript

def func():
    return Javascript("IPython.notebook.kernel.execute(`notebook_name = '${IPython.notebook.notebook_name}'`)")
func()

The problem is that this code is executed asynchronously and I do not have the result until all cells have finished executing whereas I need the result much before. This creates a problem when I am trying to do a Run all cells or %run notebook.ipynb from another notebook where the notebook.ipynb file holds this js code. I tried using

%block func

but it doesn't work.

Help is much obliged.

Kernel.do_one_iteration never awaited

Ran into this while working on ideas to solve #10. Was on WSL / dockerized Jupyter based off the docker-stacks minimal-notebook image.

Code:

# cell 1
import ipython_blocking
import ipywidgets as widgets

button = widgets.Button(description="click me", button_style="info")
button

# cell 2
%blockrun button

Error:

/home/jovyan/ipython_blocking/ipython_blocking/ipython_blocking.py:21: RuntimeWarning: coroutine 'Kernel.do_one_iteration' was never awaited
  self.kernel.do_one_iteration()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Versions:

jupyter core     : 4.7.1
jupyter-notebook : 6.4.0
qtconsole        : not installed
ipython          : 7.25.0
ipykernel        : 6.0.3
jupyter client   : 6.1.12
jupyter lab      : 3.0.16
nbconvert        : 6.1.0
ipywidgets       : 7.6.3
nbformat         : 5.1.3
traitlets        : 5.0.5

After %blockrun <button>, notebook jumps to bottom

Hi,

Functionality-wise, I've been looking for this exact functionality for a while, so thanks for making this so easy with one line of code!!

One thing I've noticed though is that after the %blockrun cell executes, it immediately jumps to the end of the notebook. However the reason I use this is to wait for ipywidgets input. This means the user has to scroll all the way back up to the input cells to actually enter input. Do you know if there is a way to force the web page to "stop" at the %blockrun cell so that users can enter the input right then and there?

Thanks :)

[Cross-Posted] ipyfilechooser + ipython_blocking = large memory leak

When I:

1. Run my example from https://github.com/bryceschober/ipyfilechooser_blocking_example in Jupyter or VS Code or even the binder.org link on that example repository.
2. Choose to run all cells instead of one at a time.
3. Observe a large continuous memory leak until you actually select a file and click select, and then memory usage returns to normal.

Input prompt numbering in JupyterLab not correct

I realized that in JupyterLab, the input prompt does not switch from [*] to [n] after cell execution does not block anymore. I set up this minimal example to demonstrate on Binder: https://github.com/dokempf/ipython-blocking-mwe

• Run in notebook (works):
• Run in Lab (does not work):

ImportError: cannot import name 'just_run' from 'nbclient.util'

Due to a newer version of nbclient, 'just run' doen't exist anymore.

How To Import into Jupyter Notebook

Hello @kafonek ! Great work with this project-this is exactly what I'm looking for! However, I'm pretty new with using the online Jupyter Notebook interface, and I'm not sure what I'm missing but the ipython_blocking module is not being found. Here is the current cell I am running:

import os
import sys
from IPython.display import display
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)
import ipywidgets as widgets
import ipython_blocking 

However, I get an error saying:

ImportError: No module named 'ipython_blocking'

I can't seem to find an answer to this online-any suggestions? Thank you in advance!

Not working in JupyterLab or Visual Studio Code (only in hosted Jupyter Notebook)

I find this module super useful, very elegant and quick solution, but can't get it to work in Jupyter Lab (<IPython.core.display.Javascript object> error in console) or Visual Studio Code (no errors).

Do you have any clue what I could try?

I’m trying to use it within voila

I was not able get it to work.
Is it possible to make it rerun a specific cell instead of blocking?