Consider an org-mode frontend via ob-ipython

[dangirsh]

Org mode has lots to offer over Jupyer notebooks / JupyterLab / EIN. Off the top of my head:

• good markup (markdown has issues)
• support for passing data between code blocks in different languages
• remote execution configured per-cell
• tree editing / navigation facilities (e.g. navi-mode, Speed keys, org-refile) to ease working with large documents

Code cells in Org mode can execute in Jupyter kernels (for various languages) via the ob-ipython package. This gives the user:

• dynamic completion from the kernel
• doc lookup from the kernel
• access to remote kernels

By virtue of using Org, "notebooks" written this way:

• are trivially version controlled
• can generate source via tangling - no more copy/pasting from notebooks
• are exportable to an absurd degree. Exporters include ipynb, Markdown, html with Bootstrap, LaTex/PDF, reveal.js, and Google slides. Many more (even docx) are available via tools like ox-pandoc .

The scimax project has very interesting additions (video demo) to ob-ipython, including:

• emulating most Jupyter notebook's commands + their keybindings (see the scimax-jupyter-command-mode hydra in here)
• multiple kernels in one Org file
• live editable LaTeX previews

Here are some ways Pynt could be more powerful with Org:

1. Org manages sessions (kernels) per-document, per-subtree, and per-cell, with the most specific taking precedence. This means that any operations which spawn new kernels could just inject a new subtree into the current document. You could then capture the complete history with multiple code variants attached to different kernels, all within one document.
2. Features like branch unpacking could benefit from having powerful tree manipulation facilities, which are the heart of Org. I assume you're relying on the markdown headings to manage the tree structure in the ipynb. According to this spec, you're limited to a depth of 6. Org has no limit that I can find and, more importantly, it gives the user powerful tools to navigate the tree. These tools open the door to more sophisticated cell generation facilities. For example, imagine generating a cell for each intermediate definition within a function trace of depth 5. Maybe I'd like to do that for 10 different invocations and then poke around in the tree of ~50 generated cells. Being forced to work at the top-level of a notebook would make this unwieldy, if not impossible.
3. Future proof. The ideas in Pynt are not Jupyter-specific, and ipynb can be generated from Org files ;)

I don't know the Pynt code well enough to guess the level of effort for this. It likely mainly depends on how much experience the contributor has with automating Org.

I thought the best first step would be to discuss the pros/cons here.

[ebanner]

Thanks for the well-articulated points @dangirsh! Indeed, I do believe using org for an intermediate "cell-representation" layer would be more flexible/powerful for all the reasons you mentioned. As an example, it would be cool to navigate code in a similar way one navigates an org-mode tree (i.e. expanding and collapsing sub-trees).

However, this set of changes is clearly targeted at improving the emacs side of pynt. Unfortunately, I find myself using emacs less and less these days and am focusing my efforts more on the jupyter side of things (e.g. via a jupyterlab plugin). Coupled with the fact that I haven't nailed down the use case for pynt yet (i.e. for what kinds of programs does pynt actually improve the coding experience?), someone else will likely have to lead this effort.

Features like branch unpacking could benefit from having powerful tree manipulation facilities, which are the heart of Org. I assume you're relying on the markdown headings to manage the tree structure in the ipynb. According to this spec, you're limited to a depth of 6.

This sounds like a concern below the level that pynt works at. pynt actually just "flattens" out a code region that you give it, pushing everything to the global scope. So there is no more tree structure (among nodes) in a pynt-generated EIN notebook.