Feature request: selectively remove a particular cell when converting from `.md` to `.ipynb` about jupytext

[mwouts]

Hi @agriyakhetarpal , thank you for opening this issue, and sharing this use case with such level of detail!

I will be happy to look further into this, and right now I only have time to give a few global comments, but this is what came to my mind:

1. Thanks for the links, I'll take more time to explore them, indeed JupyterLite is promising and it's really great to see it loading so fast (but I don't know yet how to make it works with Jupytext)
2. Is it possible that the code that you want to filter from the notebook would be at a better place somewhere else, e.g. in a Sphinx or a Jupyter Book plugin (if such a concept exists, I am not directly familiar with Sphinx) ?
3. In Jupytext we put a strong emphasis on the round trip conversion so there is no option to remove any cell inputs, but...
4. I would be happy to help you write a short Python script that removes some specific cells (see below)
5. Maybe you could also share your use case with the MyST-NB developers (see also 2. above)

Now re a script that would remove the cells with an "ignore" tag:

import nbformat

notebook = nbformat.read('notebook_with_an_ignore_tag_in_some_cells.ipynb', as_version=4)
notebook.cells = [cell for cell in notebook.cells if 'ignore' not in cell.metadata.get('tags', [])]
nbformat.write(notebook, 'filtered_notebook.ipynb')

[agriyakhetarpal]

Hi @agriyakhetarpal , thank you for opening this issue, and sharing this use case with such level of detail!

Thank you for a warm-hearted response to my query, @mwouts!

Thanks for the links, I'll take more time to explore them, indeed JupyterLite is promising and it's really great to see it loading so fast (but I don't know yet how to make it works with Jupytext)

Ah, JupyterLite is most likely embedding the interface for JupyterLab in a web browser, which already supports Jupytext notebooks. The kernel is WASM-powered; that is the difference more or less (I might not be completely correct about this, however).

Is it possible that the code that you want to filter from the notebook would be at a better place somewhere else, e.g. in a Sphinx or a Jupyter Book plugin (if such a concept exists, I am not directly familiar with Sphinx) ?

Maybe you could also share your use case with the MyST-NB developers

It should be possible to include the notebook from a different location or pair up the notebook from elsewhere. However, that would most likely require duplicating the contents of the notebook, and then inserting a cell that contains the directives for it to be able to host the notebook onto the webpage – I do not think that would be trivial. Interacting with the MyST-NB developers would be a good idea, sure!

In Jupytext we put a strong emphasis on the round trip conversion so there is no option to remove any cell inputs, but...

I understand, I didn't think about that aspect at all when writing this even when I knew it – based on that, this feature request could also be a bit out of scope, considering that this is a pretty unique one, haha! Please feel free to close this if you feel so, or keep it open for visibility out of the chance that someone else might need this feature someday.

I would be happy to help you write a short Python script that removes some specific cells (see below)

Now re a script that would remove the cells with an "ignore" tag:

I was able to manage loading this short snippet (with a minor change/fix) as a very minimal Sphinx extension coupled with a subprocess – here's a brief in case it would be interesting to other readers:

Tap to expand code

from sphinx.application import Sphinx
from pathlib import Path

HERE = Path(__file__).parent

def preprocess_notebooks(app: Sphinx, *args, **kwargs):
    """Preprocess notebooks to convert them to IPyNB and remove Sphinx directives."""
    import subprocess
    import sys

    import nbformat

    print("Converting Markdown files to IPyNB...")
    subprocess.check_call(
        [
            sys.executable,
            "-m",
            "jupytext",
            "--to",
            "ipynb",
            f"{HERE / '*.md'}",
        ]
    )

    for notebook in Path(HERE).glob('*.ipynb'):
        print(f"Removing Sphinx directives from {notebook}...")
        converted_notebook = nbformat.read(notebook, as_version=4)
        converted_notebook.cells = [
            cell for cell in converted_notebook.cells
            if "true" not in cell.metadata.get("ignore", [])
        ]
        print(f"Removed Sphinx directives from {notebook}.")
        nbformat.write(converted_notebook, notebook)


def setup(app):
    app.connect("builder-inited", preprocess_notebooks)

and it works perfectly (edit: there is another error, which is unrelated to this process and is more about how the notebooks are copied to the built docs). Thank you!