Lessons learned from updating Clav

S. Gilles

2018-02-18

Background

  About a year and a bit ago, I started looking at quivers, which
  are graphs that carry a cluster algebra structure. I did a bit
  of work by drawing them with pen and paper, or on a blackboard
  if one was nearby, but I wanted to click and drag nodes, and I
  wanted a computer to do the mutation arithmetic for me.

  The primary tool for this was Keller's applet [0], but it doesn't
  quite mesh with how I want to view quivers. In particular, vertices
  can't be renamed and can't carry non-trivial weights. Plus, it's
  written in Java, and my hourly price for working with Java is
  about three times what the university is currently paying me.

  So I wrote Clav [1], added the features I needed, and then stopped
  looking at the source code for quite a while. About two weeks
  ago, office-mates who were also looking at cluster algebras
  expressed interest in Clav. After a few days of using it, they
  had some feature requests. I took a shot at implementing them,
  after not having really thought much about the code for 8 months
  or so. These are my reflections on that work.

Clav's basic structure

  The mathematical operations for Clav are entirely stored in
  quiver.c. The main() function lives in clav.c, and consists
  entirely of sending repaint instructions to the UI (there's also
  a CLI component, so the UI is modular) and passing along the UI's
  requests to quiver.c.

  I enforced the UI modularity early on, in order to force myself
  to separate mathematic code from display code. I think the result
  might be MVC, but I prefer to call my design paradigm “making
  sure the names of the files reflect what's in them”.

  The UI is explicitly a medium-sized state machine, with states
  like “WAITING FOR ACTION TO BE SELECTED” and “ENTERING TEXT WHICH
  WILL BE USED AS NEW NAME OF VERTEX”. This is in contrast to
  previous projects I've worked on, which work by swapping out
  Controller objects and function pointers.

  I had been thinking on a particular essay [2] of Carmack's, so
  there are huge switch statements, and few helper functions in
  Clav; where things can be inlined without introducing repetition,
  they usually are.

  The file selection dialogs for saving and loading are messier
  than the rest of the code (more on that later), so the code dealing
  with them is quarantined.

Clav 0.4

  The users' complaints about the UI were the following:

   o Actions on the quiver (adding vertices, edges, performing
     mutations) typically happen in chunks. The program logic flow
     was a boring state machine

       base state ---(input to select action)---> action state
         ^                                          |
         |                                          |
         +-- perform action <---(input for action)--+

    So adding ten vertices required pressing ‘v’ ten times, and
    clicking ten times, all intermingled.

   o Keller's applet lets you change colors, Clav didn't. I'm used
     to the individual vertices of quivers being important, so I
     care about names. If you're only interested in the general
     shape of the graph, it's useful to have a more obvious
     distinguisher, like color.

   o No undo buffer.

  Interestingly, they didn't mind the following quirks of the UI:

   o No menus. The whole application is SDL-only because after
     evaluating Gtk, Qt, EFL, FLTK, IUP, Motif, Tk, Dear ImGui,
     wxWidgets, and probably some others, I can't say I like any
     widget toolkit out there today.

     As a result, the program feels like a TUI that happens to use
     graphics. To perform an action, a user presses the relevant
     key, then clicks the mouse for a bit.

   o Learning curve. My programs, written for me, usually have such
     poor discoverabilty that they're impossible for others to use.
     For Clav, however, I decided that any time the program was
     running, it would show the user (in text) keys for every action
     available. This seems to have worked.

   o Saving and loading files (because I didn't want to commit to
     a widget toolkit) work by trying various stand-alone dialog
     programs, like Qarma or Zenity. Because multi-threading is
     hard, I decided not to try it. As a result, Clav will completely
     block when such a prompt is open. Nobody seemed to notice this,
     however.

  One design choice I had made turned out to be well-received:

   o Store saved data files as boring plaintext files with a structure
     that can probably be guessed at by examining one. They have
     some programs of their own, and were hoping to use Clav as a
     quick UI to interact with the outputs of their calculations.
     It looks like that will be possible.

Clav 1.0

  I decided to address two of the feature requests: repeated actions
  and color changing. The undo buffer should be technically possible
  (cluster algebras are involutions), but there seems to be exactly
  one use case at the moment (fiddling around to make Q₁ look like
  Q₂, then replaying the mutation sequence), which may indicate
  that there's a more useful modification that could be made.

  Repeated actions were surprisingly clean to implement. The explicit
  state machine was reasonable to modify; instead of unconditionally
  switching to UIA_NONE when UIA_NEW_EDGE finishs, make it conditional
  on whether SDL reported KMOD_SHIFT. I don't think my choice to
  explicitly encode the state machine caused this, but rather my
  choice to be consistent and disciplined about making all actions
  follow the same pattern — any pattern.

  Colors were also surprisingly easy; I didn't think they'd be
  possible at all. It turns out Qarma, Zenity, &c (even the lowly
  Xdialog) support selection of colors, so I was able to mostly
  duplicate my file selection pattern: open a third-party program
  via popen(), read the results, and try not to worry that the main
  window is blocked from repainting.

An Object Lesson in YAGNI

  My decision to quarantine the dialog-opening code seems to have
  paid off: I was able to read all of file-selection.c quickly,
  learn/remember what I was thinking, and re-apply that to another
  type of dialog.

  I was reminded that, during the initial process of writing Clav,
  I was thinking how I'd need to expand dialog handling later. My
  thoughts, however, were always on expanding the number of supported
  third-party dialog programs. If I had given into the Impulse To
  Generalize, I would have ended up with an auto-detected subdirectory
  of files: one for Qarma, one for Zenity, one for Xdialog, and all
  for opening file selection windows. (This is not hypothetical; I
  was halfway into the necessary Makefile bits before common sense
  told me to stop.)

  Updating that infrastructure to deal with color selection would
  have been a pain in the neck, since color selection adds input
  parsing requirements. To choose a filename to load, no parsing
  is needed: every program spits out “/path/to/foo.txt” on stdout,
  which is exactly the form needed for use by ui-sdl.c. Color
  selection, however, needs parsing of “#rrggbb” or “rrr ggg bbb”
  to be useful. Some programs output one form, but they aren't all
  unified. My over-automated, generalized infrastructure for pluggable
  dialog generators would have stumbled on that, because it wasn't
  generalized in the correct direction.

Myrddin

  Most of my programming work over the last 1.5 years or so has
  been in Myrddin [3], which I think is probably the best C-type
  language today. As I was working with Clav again, I noticed the
  following:

   o I missed being able to return algebraic types more than I
     thought I would. It would also have relieved a couple of pain
     points for me in C (the WIFEXITED dance for pclose(), most
     string handling).

   o Switching on algebraic data types would make state transitions
     much more pleasant. A gripe I've had many times, but no less
     true.

   o Myrddin's __disposable__ trait has allowed me to replace the
     “goto done;” dance with annotation of expressions that need
     to be cleaned up at return time. This would simplify a lot of
     bits in Clav.

   o Growing arrays (slices) is much easier in Myrddin, but

   o I have much finer control over error conditions (e.g. checking
     for overflow in malloc(a * b)) in C. Most of that isn't needed
     in Myrddin… ‘most’. Someone will need to address the “abort
     on ENOMEM” thing in Myrddin one day.

Conclusions:

  Clav has, at least this time, not failed the “Put it down for a
  year, then add a major feature” test.

   o I don't attribute this necessarily to good code, but to
     consistent style, probably caused because it was written by a
     single developer in a short period of time.

   o Long functions that don't fit in an 80x24 window aren't poison.
     It was necessary for me to reread and relearn a few, but I
     never had to open another editor/tab/pane/whatever and break
     my concentration to learn a helper function.

   o In the vein of the above, I never had to worry “If I change
     this line, which users will be affected?”, although grep would
     have solved that pretty quickly. Functions were either obviously
     general or obviously single-purpose.

   o I have more confirmation that explicit state machines are
     adequate for low- to mid-complexity UI work. I'll probably use
     this pattern going forward.

   o Speaking for myself only, I really Ain't Gonna Need It. Or,
     at least, I needed something different that the premature
     generalization would have made painful.

[0] https://webusers.imj-prg.fr/~bernhard.keller/quivermutation/
[1] git://repo.or.cz/llf.git
[2] http://archive.is/Pp1Gf
[3] https://myrlang.org/