============================================================= gopher applet spec — someodd.zip ============================================================= A "gopher applet" on this server is a literate Haskell script that runs server-side when its selector is requested, and emits its output as the gopher response. Drop one into /applets/ and it executes on first hit, the compiled binary is cached, and every subsequent request is near-instant (~50 ms) until the source is edited. ------------------------------------------------------------- in the wild ------------------------------------------------------------- Check out my applets: gopher://gopher.someodd.zip/1/applets/ All of my currently served gopher applet's source is here: gopher://gopher.someodd.zip/1/applets_source/ ------------------------------------------------------------- what you get ------------------------------------------------------------- This is one of those rare points where three independent tools — Venusia, bartleby, and Haskell's literate-script mode — combine into something more than their parts. A working applet is, in one .lhs file: * A markdown document you can read top-to-bottom. Bird- track literate Haskell is prose by default; lines starting with '> ' are code. Open it in any text editor; render it with any markdown renderer; the source IS the documentation. * Self-contained dependencies. One -- stack script --resolver lts-22.6 --package text line at the top is all the build config you need. No package.yaml, no .cabal, no separate lockfile. * Reproducible. Stack's resolver pins the GHC version and every transitive dependency. The same .lhs compiles the same way today and a year from now, until you bump the resolver on purpose. * Fast at runtime. run-cached-lhs hashes the source's mtime+size and caches the compiled binary under /var/cache/venusia-compiled. First request after an edit pays the compile cost; every request after is ~50 ms of bash + exec. The 2-3 second stack startup that would otherwise hit every menu navigation is amortised away. * Mounted by config, not code. Where the applet lives in gopherspace — its selector, its host, its file path — is a routes.toml decision. Move the .lhs to a different [[files]] block; the script keeps working because it learns its selector at request time via $selector. * Path-info dispatch for virtual sub-trees. One [[files.script_extension]] block lets one .lhs back an entire menu tree: /wiki.lhs/Page/SubPage, /figlet.lhs/banner, /api.lhs/v1/users. The daemon hands the script the path-info portion as $pathinfo and the script branches on it. * Inline tests for pure functions. Haddock '>>>' examples in front of any pure function are verified by `doctest`, invoked via the `doctest-lhs ` wrapper (peer to run-cached-lhs; reuses the script's own `-- stack` directive). No separate test file, no harness setup — the examples live next to the function they document. * Catalog integration for free. bartleby walks the source tree, reads each .lhs and any sibling .bcard for metadata, and emits .gophermap files that link your applet with the right item type. You don't register the applet anywhere — bartleby finds it. * Plain text on the wire. A .lhs applet's output is a gophermap as raw bytes. `nc gopher.someodd.zip 70` followed by the selector shows you exactly what your script produced. No serialisation layer, no opaque transport, no debugger needed for "what did this emit". The combination is unusually pleasant for the moving-parts count: prose documentation, runnable code, declarative dependencies, typed verification, fast-by-default deployment, and a zero-config catalog wrapper — in a single file you can email to someone. ------------------------------------------------------------- the convention ------------------------------------------------------------- A file at /var/gopher/output/applets/.lhs: * is literate Haskell — code lines start with '>' * declares its dependencies on a literate-marker line near the top: #!/usr/bin/env stack > -- stack script --resolver lts-22.6 --package text \ > --package process The stack directive lives on a '>' line because stack 3.x's interpreter mode reads it from there for .lhs files. * receives four CLI arguments: 1. the gopher selector that resolved to it (lets a script emit self-links without hardcoding its path) 2. the request's search query (after the tab; empty when the request had none) 3. the path-info: the selector portion AFTER the script's filename, leading slash included, empty when the script was addressed directly. A request for /applets/wiki.lhs/topics/gopher gives path-info "/topics/gopher". Lets one applet back a virtual sub-tree of pages by branching on path-info segments. 4. the client's IP address as text (IPv4 dotted-quad or IPv6 colon form); empty when the peer can't be looked up. Useful for rate limiting, per-IP rules, or audit logging — the script decides what to do with it. Use a cons pattern (`(s:q:p:ip:_) -> ...`) when matching argv so future arg additions don't break your script. * has a "front door": the response it emits at its bare selector — empty path-info (arg 3 above), before the visitor has navigated into any sub-path. For a path-info-dispatching applet (one .lhs backing a virtual sub-tree) the front door is the entry page you arrive at first: an explainer, a menu, a create-link. ("Landing page" is the web habit, but gopherspace is navigated as places — you go to a hole, walk a menu — so "front door" fits the spatial metaphor and keeps web-era vocabulary out of a pre-web protocol.) * writes its response to stdout. Either: - a complete gophermap (info lines, search items, ending with ".\r\n"), or - raw bytes for a binary type-9 response (audio, images, archives, whatever) * (optional) if the file opts into verification — LH refinements, `>>>` doctest examples, or both — carries a short top-of-file prose section naming the commands an author runs to exercise each half. The runners are framework (doctest-lhs is peer to run-cached-lhs; LH rides inside the compile), but the .lhs documents how to verify itself. See `grep.lhs` for the worked example. ------------------------------------------------------------- how it stays fast ------------------------------------------------------------- The daemon (Venusia) routes /applets/*.lhs through a wrapper at /usr/local/bin/run-cached-lhs. The wrapper: 1. hashes the source file's mtime + size 2. if /var/cache/venusia-compiled/. exists, execs it directly — no stack startup 3. otherwise compiles via 'stack ghc' with the directive's packages, writes the binary atomically, then execs it First request after editing: pays the compile cost (seconds for boot-lib-only scripts, minutes for ones that pull non-boot dependency trees like http-client). Every request after: ~50 ms of bash + exec overhead, then the script's actual work. This means stack is touched once per source edit, not once per request. The 2-3 second stack startup that would otherwise hit every menu navigation is amortised away. ------------------------------------------------------------- how it surfaces in the catalog ------------------------------------------------------------- The menu tree this server hands clients is generated by bartleby — a gopher catalog generator that walks /var/gopher/output/, reads each file plus an optional sibling .bcard for metadata, and emits catalog/.gophermap files Venusia serves. The source tree is read-only to bartleby; only catalog/ is generated, and deleting it leaves your library exactly as you wrote it. By default bartleby treats an unknown extension as a text file (item type 0). A line in bartleby.conf tells it to mark .lhs entries as menus (item type 1) instead, so a client navigating the catalog sees them as gopher menus and follows them without being prompted for input: item_types: .lhs: "1" bartleby >= 0.3.1 is required — earlier versions rejected '1' as an item_types value on the theory that menus must be directories. v0.3.1 lifted that exactly because CGI-style script files want to declare "I am the menu" to the catalog. Optional metadata: a sibling .lhs.bcard with `title:` / `description:` / `created:` / `updated:` shows up under the applet's catalog entry. Without one, the catalog entry is just the bare filename. Pipeline, end to end: 1. you drop applets/figlet.lhs (and optionally applets/figlet.lhs.bcard) into /var/gopher/output/ 2. bartleby reads the tree and writes catalog/applets/.gophermap with a line: 1figlet text renderer/applets/figlet.lhshostport because of the .lhs->1 mapping in bartleby.conf 3. a client navigating the catalog clicks that entry — its type-1 prefix means "menu", so the client follows without prompting 4. Venusia matches /applets/figlet.lhs against its [[files.script_extension]] rule, run-cached-lhs execs the (cached) compiled binary, the binary writes a gophermap to stdout, and Venusia returns that verbatim bartleby is at https://github.com/someodd/bartleby. ------------------------------------------------------------- setting up a host ------------------------------------------------------------- Three things have to be on the box before an applet can serve: * `stack`, on the serving user's PATH. The path of least resistance is ghcup (https://www.haskell.org/ghcup/), the official Haskell toolchain installer — it drops stack, GHC, and cabal under ~/.ghcup/bin. You don't pick or manage a GHC version by hand: each script pins its own resolver on the `-- stack` line, and stack fetches the matching GHC the first time that resolver compiles. ghcup's job here is just to get `stack` itself onto the system. * the run-cached-lhs and doctest-lhs wrappers on PATH, conventionally /usr/local/bin. Both ship in this directory: sudo install -m 755 run-cached-lhs doctest-lhs /usr/local/bin/ * the Venusia daemon, with a routes.toml that routes /applets/*.lhs through run-cached-lhs. The routes.toml in this directory is the working example; the [[files.script_extension]] block is the part that does it. One more, but only for applets that opt into LiquidHaskell: the z3 CLI (`apt install z3`) and a resolver where `liquidhaskell` resolves. Both are detailed under "optional: liquidhaskell verification" below — applets without LH bits need neither. ------------------------------------------------------------- minimal template ------------------------------------------------------------- #!/usr/bin/env stack > -- stack script --resolver lts-22.6 --package text hello.lhs — emit a one-line gophermap that names the selector that resolved to this script. > {-# LANGUAGE OverloadedStrings #-} > module Main (main) where > > import qualified Data.Text as T > import qualified Data.Text.IO as TIO > import System.Environment (getArgs) > > main :: IO () > main = do > args <- getArgs > let sel = case args of (s:_) -> T.pack s > _ -> "/applets/hello.lhs" > TIO.putStr ("iHello from " <> sel <> "\t\t\t0\r\n") > TIO.putStr ".\r\n" ------------------------------------------------------------- running it locally ------------------------------------------------------------- The `#!/usr/bin/env stack` shebang on every applet means the file is directly executable once you set the execute bit: chmod +x hello.lhs ./hello.lhs stack reads the `-- stack` directive, resolves the resolver and packages, compiles the source (or reuses its own script-binary cache under ~/.stack/script-binaries/ if nothing has changed), and execs the binary. Same compile, same LH refinement checks if the file enables the plugin, same gophermap on stdout. This is the path you work in while writing an applet — edit, save, run, read the bytes, repeat — with no Venusia install in the loop. It's slower per run than the served path, though: stack revalidates its cache on every invocation, which costs the 2-3 seconds run-cached-lhs exists to amortise away. The wrapper caches the binary at a known location keyed by mtime+size and execs it directly, never waking stack on a hit, so /applets/ requests stay at ~50 ms. The shebang path is the same applet end to end — it just pays the startup tax each time in exchange for needing nothing but `stack` on PATH. That PATH caveat is the one gotcha: a non-interactive ssh session (no `.zshrc` sourced) won't find stack by default. Run as `PATH=/home/venusia/.ghcup/bin:$PATH ./script.lhs`, or log in interactively first. ------------------------------------------------------------- running doctests ------------------------------------------------------------- Doctest is a post-compile harness: it spawns GHCi, reloads the file, evaluates each `>>>` example, and diffs the result against the expected output line below. That makes it a separate step from the compile itself — distinct from the LiquidHaskell pipeline below, which lives inside the compile as a GHC plugin and so fires automatically every time the file is built. The two halves are complementary: doctest covers `>>>` examples on pure helpers, LH covers refinement types; together they form the file's verification path. The companion wrapper to run-cached-lhs: doctest-lhs path/to/script.lhs It reads the script's own "-- stack" directive (so the resolver and --package list always match what run-cached-lhs compiles against) and extracts the file's LANGUAGE pragmas as -X flags for doctest's GHCi session. No per-script invocation prose to maintain; no -XOverloadedStrings to remember. For scripts that enable the LiquidHaskell plugin (see next section), the wrapper rewrites the plugin pragma to an inert comment in a temp copy before invoking doctest, so LH's "LIQUID: SAFE ..." banner doesn't poison doctest's stdout capture. Line numbers are preserved; doctest failures still point at the right source line. Exit status mirrors doctest's: 0 on a clean run, non-zero on any failure. The wrapper lives at /usr/local/bin/doctest-lhs alongside run-cached-lhs and uses the same directive-parsing awk so the two stay in step. ------------------------------------------------------------- optional: liquidhaskell verification ------------------------------------------------------------- For applets that touch untrusted input or carry size/range invariants, you can ask LiquidHaskell to check pure-function refinements at compile time. Unlike doctest above, LH is a GHC plugin: enabling it adds a check inside the compile that run-cached-lhs (or the shebang path) runs anyway, so there is no separate command — every cache miss re-runs the refinements, and a failure prevents the binary from being cached at all. The integration is opt-in per file — the minimal template above doesn't carry any LH bits, and applets that don't want refinement types keep working unchanged. Host requirements (one-time): * `z3` CLI installed (`sudo apt install z3`). The Z3 *library* alone isn't enough — LH shells out to the binary at build time. * A resolver where `liquidhaskell` resolves. As of writing, no `lts-22/23/24` snapshot ships it; a date-pinned `nightly` is the practical choice. The author tested `nightly-2025-12-01` (GHC 9.12.2 + liquidhaskell 0.9.12.2) and it works end-to-end with run-cached-lhs. Per-file opt-in (three lines added to the script): 1. The `--package liquidhaskell` flag on the `-- stack` directive line. 2. The plugin pragma: {-# OPTIONS_GHC -fplugin=LiquidHaskell #-} 3. Whichever LIQUID option lines you need to silence LH's stricter defaults. For most retrofits onto existing IO- containing code: {-@ LIQUID "--no-totality" @-} {-@ LIQUID "--no-termination" @-} Then write the refinements you want as `{-@ ... @-}` blocks above the relevant functions. Three flavours, ordered from least to most useful: * Numeric/range refinements on top-level constants (`{-@ maxSnippetLen :: {v:Int | v >= 4} @-}`) — LH *verifies* these. Drop the constant below the bound and compilation fails. * `assume` annotations declare a trusted postcondition LH won't verify but will propagate to callers. Useful for small audited functions like a sanitiser, where the body is obvious and the win is forcing every call site to admit it. * Function refinements with `{-@ name :: ... @-}` (no `assume`) — LH *verifies* these by checking the body against the spec. Most powerful, most fragile; the library's measures (e.g. `len` on lists) cover lists cleanly but not `Data.Text` in LH 0.9.12. Failure visibility: LH errors come through the same path as GHC errors. run-cached-lhs's `set -eu` aborts before the binary is cached, the request's stderr surfaces the error, and the client sees no body. Same shape, same UX, same debugging path as a type error. First-compile cost: LH + Z3 + the LH dep tree compile takes ~15-30 min on a small server the first time the resolver is seen. Subsequent compiles on the same resolver are cached at the stack-snapshot level; subsequent requests against the same source-mtime hit the run-cached-lhs binary cache (~50 ms). The slow path is paid once per resolver bump, not per request and not per source edit. LH and the two invocation paths: LH runs at compile time, not runtime. The cached binary that ends up under /var/cache/venusia-compiled/ is ordinary Haskell — executing it doesn't re-check the refinements. That's not a hole; it's how typed languages work. The check that matters happens before the binary is written. LH fires on both compile paths — the daemon's run-cached-lhs invocation and the shebang path (see "running it locally" above) — checking on first compile and skipping on unchanged source. The two caches are independent, though: a fix you've re-exercised via the shebang doesn't refresh the daemon's binary under /var/cache/venusia-compiled/, which stays stale until the next request recompiles. When in doubt about which binary is live, edit-and-touch the source to force both. `grep.lhs` in this directory is the worked example. ------------------------------------------------------------- verified template ------------------------------------------------------------- #!/usr/bin/env stack > -- stack script --resolver nightly-2025-12-01 --package text --package liquidhaskell verified-hello.lhs — same as the minimal template, but with one verified refinement type and one doctest example, to prove both halves of the verification pipeline are alive. > {-# LANGUAGE OverloadedStrings #-} > {-# OPTIONS_GHC -fplugin=LiquidHaskell #-} > > {-@ LIQUID "--no-totality" @-} > {-@ LIQUID "--no-termination" @-} > > module Main (main) where > > import qualified Data.Text as T > import qualified Data.Text.IO as TIO > import System.Environment (getArgs) > > -- | The number of rows we emit. LH verifies the @>= 1@ > -- refinement at compile time; flip the body to 0 and > -- compilation fails before the binary is cached. The > -- @>>>@ example below is verified by `doctest-lhs`. > -- > -- >>> rowCount > -- 1 > {-@ rowCount :: {v:Int | v >= 1} @-} > rowCount :: Int > rowCount = 1 > > main :: IO () > main = do > args <- getArgs > let sel = case args of (s:_) -> T.pack s > _ -> "/applets/verified-hello.lhs" > TIO.putStr ("iHello from " <> sel > <> " (" <> T.pack (show rowCount) <> " row)\t\t\t0\r\n") > TIO.putStr ".\r\n" Two commands exercise the two halves: ./verified-hello.lhs # compile -> LH refinements checked doctest-lhs verified-hello.lhs # >>> examples verified ------------------------------------------------------------- limits ------------------------------------------------------------- * Server caps in-flight connections at 256. Long-lived streams (icecast.lhs, future radio things) count. * Accepted sockets have Linux TCP_USER_TIMEOUT = 120s; a silently-dead client is reaped within two minutes. * The compile cache survives reboots. After a GHC version upgrade (ghcup install ghc ), the dynamically-linked binaries break; clear with: sudo rm -rf /var/cache/venusia-compiled/* * Each script's output should be self-contained (gophermap terminator if menu, raw bytes if binary). The framework passes stdout through verbatim with no transformation. ------------------------------------------------------------- see also ------------------------------------------------------------- The daemon is Venusia. The relevant features are nested [[files.script_extension]] (0.7.0.0), $selector substitution (0.7.0.0), trailing-slash routing equivalence (0.7.1.0), $pathinfo path-info dispatch (0.8.0.0), $remote_ip peer-IP substitution (0.10.0.0), and the streaming response constructor (0.6.0.0). https://github.com/someodd/venusia The catalog these applets surface under is generated by bartleby. v0.3.1 added the `item_types: .lhs: "1"` mapping that makes .lhs catalog entries link as menus rather than as text files — without it, clicking a .lhs row would prompt the client to fetch raw source. https://github.com/someodd/bartleby To see a more complete and working set of Gopher applet source files please see this URI: