The Magnificent Maniposynth

(Download video)

Visual non-linear editing, live programming, and synthesis for (some of) OCaml.

Maniposynth is described in our paper: Brian Hempel and Ravi Chugh. Maniposynth: Bimodal Tangible Functional Programming. ECOOP 2022. This page contains the video figure (above) and the artifact (below) so you can try Maniposynth yourself.

Abstract

Traditionally, writing code is a non-graphical, abstract, and linear process. Not everyone is comfortable with this way of thinking at all times. Can programming be transformed into a graphical, concrete, non-linear activity? While nodes-and-wires and blocks-based programming environments do leverage graphical direct manipulation, users perform their manipulations on abstract syntax tree elements, which are still abstract. Is it possible to be more concrete—could users instead directly manipulate live program values to create their program?

We present a system, Maniposynth, that reimagines functional programming as a non-linear workflow where program expressions are spread on a 2D canvas. The live results of those expressions are continuously displayed and available for direct manipulation. The non-linear canvas liberates users to work out-of-order, and the live values can be interacted with via drag-and-drop. Incomplete programs are gracefully handled via hole expressions, which allow Maniposynth to offer program synthesis. Throughout the workflow, the program is valid OCaml code which the user may inspect and edit in their preferred text editor at any time.

With Maniposynth's direct manipulation features, we created 38 programs drawn from a functional data structures course. We additionally hired two professional OCaml developers to implement a subset of these programs. We report on these experiences and discuss to what degree Maniposynth meets its goals of providing a non-linear, concrete, graphical programming workflow.

Read the paper for more.

Artifact Download

You can grab the VM or a prebuilt binary. The Ubuntu VM includes the prebuilt binary and VS Code. The advantage of the VM is that VS Code is preconfigured with the OCaml language server and a special highlighting rule to dim AST annotations. The disadvantage of the VM is that it's a 1000x bigger download than the binaries. I also find graphical interaction in VirtualBox to be rather clunky, even with hardware acceleration on. And the VS Code configuration is entirely optional, Maniposynth works with any text editor that will refresh when the file changes on disk (see below for how to enable this in Vim/Emacs if you prefer those editors).

Pre-built binaries (~5MB, effectively what we gave to user study participants):

VM Image (with VS Code configured)

Build from source (if the above fail)

Quickstart

If you dowloaded the VM, there's an "Open Maniposynth" shortcut on the desktop that will start the Maniposynth server, open the artifact folder in VS Code, and open Maniposynth in Chromium on a scratch.ml file.

For the pre-built binaries, enter the artifact folder and start the server. (On Mac, to trust the executable, you will have to right-click the executable and select "Open". After that first time, you can start it from the command line as normal.)

Then open Chrome and navigate to a file path relative to the server, e.g. http://localhost:1111/examples/length.ml. You can also create a new file (in your editor) and type that file path in the URL, e.g. http://localhost:1111/scratch.ml.

Open up the same file in your editor, side by side, for the full bimodal experience.

If you have ocamlformat installed and in your $PATH, generated code will have better formatting, but this is not required.

Live Refresh

VS Code will automatically refresh when code is changed on disk. For Vim and Emacs, see below:

Vim

Thanks to eli on Super User for this solution. Run this in Vim:

After cursor stops moving, this will check every updatetime seconds for file changes, which is every 4 seconds by default. (More specifically, Vim waits for the cursor to stop moving for some time, then checks disk, then moves the cursor again with “lh” to retrigger and loop.)

To poll every half second instead of every 4 seconds:

If you get annoying bells, turn Vim’s bell off:

Emacs

For Emacs, enable global-auto-revert-mode:

  1. Hit F10 to go to the top menu
  2. Options > Customize Emacs > Specific Option
  3. Type global-auto-revert-mode
  4. Move the cursor to [ Toggle ] and hit Enter
  5. Move to [ Apply and Save ] and hit Enter

Dimming AST Annotations (Optional)

To dim AST attribute annotations to make code more readable, in VS Code install the Highlight extension (fabiospampinato.vscode-highlight). This artifact includes a .vscode/settings.json that contains the regex and styling for annotations. It should "just work" if you open the artifact folder in VS code (e.g. via code artifact). You may have to edit the styling in .vscode/settings.json if you use a dark theme. (The VM is already configured with the Highlight extension.)

Bugs to know

Have fun!

Artifact Evaluation Walkthrough

For artifact evaluation, we claim (a) that our artifact works as described in the Overview example in Section 2 of the paper, and (b) that our artifact works not just on the overview example.

To verify these claims, (a) we ask the artifact reviewers to follow the walkthrough in the paper Overview, including the variations, and (b) we ask the reviewers to try one example of their own choosing, either of their own design or from Table 1.

Initial variation (Sec 2.1)

Start with a blank file. (Delete everything in your text editor and save.)

Undo and delete (Sec 2.2)

This section claims that Undo works, and delete works on expressions and on let-bindings.

Drag-to-extract (Sec 2.3)

Autocomplete-to-extract (Sec 2.3)

Asserts (Sec 2.3)

Program Synthesis (Sec 2.3)

Subvisualizations, with asserts (removed from final paper)

Try your own example

Pick a simple function of your choice and try to implement it—the easier exercises in Table 1 can be an inspiration. Final code for each example in Table 1 is included in the artifact/examples folder, if that is helpful. It is okay to perform text edits in your editor if you get stuck—this is a bimodal environment after all! Hints: