Directory listing for: someodd/opensource/gopher_applet_spec
DIR Parent directory (..)
TEXT README.txt (21.6KB, 2026-05-23):
=============================================================
gopher applet spec — someodd.zip
=============================================================
A "gopher applet" on this server is a literate Haskell script
that runs server-side when its selector is requested, and emits
its output as the gopher response. Drop one into /applets/ and
it executes on first hit, the compiled binary is cached, and
every subsequent request is near-instant (~50 ms) until the
source is edited.
-------------------------------------------------------------
in the wild
-------------------------------------------------------------
Check out my applets:
gopher://gopher.someodd.zip/1/applets/
All of my currently served gopher applet's source is here:
gopher://gopher.someodd.zip/1/applets_source/
-------------------------------------------------------------
what you get
-------------------------------------------------------------
This is one of those rare points where three independent
tools — Venusia, bartleby, and Haskell's literate-script
mode — combine into something more than their parts. A
working applet is, in one .lhs file:
* A markdown document you can read top-to-bottom. Bird-
track literate Haskell is prose by default; lines
starting with '> ' are code. Open it in any text editor;
render it with any markdown renderer; the source IS the
documentation.
* Self-contained dependencies. One
-- stack script --resolver lts-22.6 --package text
line at the top is all the build config you need. No
package.yaml, no .cabal, no separate lockfile.
* Reproducible. Stack's resolver pins the GHC version and
every transitive dependency. The same .lhs compiles the
same way today and a year from now, until you bump the
resolver on purpose.
* Fast at runtime. run-cached-lhs hashes the source's
mtime+size and caches the compiled binary under
/var/cache/venusia-compiled. First request after an edit
pays the compile cost; every request after is ~50 ms of
bash + exec. The 2-3 second stack startup that would
otherwise hit every menu navigation is amortised away.
* Mounted by config, not code. Where the applet lives in
gopherspace — its selector, its host, its file path —
is a routes.toml decision. Move the .lhs to a different
[[files]] block; the script keeps working because it
learns its selector at request time via $selector.
* Path-info dispatch for virtual sub-trees. One
[[files.script_extension]] block lets one .lhs back an
entire menu tree: /wiki.lhs/Page/SubPage,
/figlet.lhs/banner, /api.lhs/v1/users. The daemon hands
the script the path-info portion as $pathinfo and the
script branches on it.
* Inline tests for pure functions. Haddock '>>>' examples
in front of any pure function are verified by `doctest`,
invoked via the `doctest-lhs <file>` wrapper (peer to
run-cached-lhs; reuses the script's own `-- stack`
directive). No separate test file, no harness setup —
the examples live next to the function they document.
* Catalog integration for free. bartleby walks the source
tree, reads each .lhs and any sibling <name>.bcard for
metadata, and emits .gophermap files that link your
applet with the right item type. You don't register the
applet anywhere — bartleby finds it.
* Plain text on the wire. A .lhs applet's output is a
gophermap as raw bytes. `nc gopher.someodd.zip 70`
followed by the selector shows you exactly what your
script produced. No serialisation layer, no opaque
transport, no debugger needed for "what did this emit".
The combination is unusually pleasant for the moving-parts
count: prose documentation, runnable code, declarative
dependencies, typed verification, fast-by-default
deployment, and a zero-config catalog wrapper — in a single
file you can email to someone.
-------------------------------------------------------------
the convention
-------------------------------------------------------------
A file at /var/gopher/output/applets/<name>.lhs:
* is literate Haskell — code lines start with '>'
* declares its dependencies on a literate-marker line near
the top:
#!/usr/bin/env stack
> -- stack script --resolver lts-22.6 --package text \
> --package process
The stack directive lives on a '>' line because stack
3.x's interpreter mode reads it from there for .lhs files.
* receives four CLI arguments:
1. the gopher selector that resolved to it (lets a
script emit self-links without hardcoding its path)
2. the request's search query (after the tab; empty when
the request had none)
3. the path-info: the selector portion AFTER the
script's filename, leading slash included, empty when
the script was addressed directly. A request for
/applets/wiki.lhs/topics/gopher gives path-info
"/topics/gopher". Lets one applet back a virtual
sub-tree of pages by branching on path-info segments.
4. the client's IP address as text (IPv4 dotted-quad or
IPv6 colon form); empty when the peer can't be looked
up. Useful for rate limiting, per-IP rules, or audit
logging — the script decides what to do with it.
Use a cons pattern (`(s:q:p:ip:_) -> ...`) when matching
argv so future arg additions don't break your script.
* has a "front door": the response it emits at its bare selector
— empty path-info (arg 3 above), before the visitor has
navigated into any sub-path. For a path-info-dispatching applet
(one .lhs backing a virtual sub-tree) the front door is the entry
page you arrive at first: an explainer, a menu, a create-link.
("Landing page" is the web habit, but gopherspace is navigated as
places — you go to a hole, walk a menu — so "front door" fits the
spatial metaphor and keeps web-era vocabulary out of a pre-web
protocol.)
* writes its response to stdout. Either:
- a complete gophermap (info lines, search items, ending
with ".\r\n"), or
- raw bytes for a binary type-9 response (audio, images,
archives, whatever)
* (optional) if the file opts into verification — LH
refinements, `>>>` doctest examples, or both — carries a
short top-of-file prose section naming the commands an
author runs to exercise each half. The runners are
framework (doctest-lhs is peer to run-cached-lhs; LH
rides inside the compile), but the .lhs documents how
to verify itself. See `grep.lhs` for the worked example.
-------------------------------------------------------------
how it stays fast
-------------------------------------------------------------
The daemon (Venusia) routes /applets/*.lhs through a wrapper at
/usr/local/bin/run-cached-lhs. The wrapper:
1. hashes the source file's mtime + size
2. if /var/cache/venusia-compiled/<name>.<hash> exists,
execs it directly — no stack startup
3. otherwise compiles via 'stack ghc' with the directive's
packages, writes the binary atomically, then execs it
First request after editing: pays the compile cost (seconds for
boot-lib-only scripts, minutes for ones that pull non-boot
dependency trees like http-client). Every request after: ~50 ms
of bash + exec overhead, then the script's actual work.
This means stack is touched once per source edit, not once per
request. The 2-3 second stack startup that would otherwise hit
every menu navigation is amortised away.
-------------------------------------------------------------
how it surfaces in the catalog
-------------------------------------------------------------
The menu tree this server hands clients is generated by
bartleby — a gopher catalog generator that walks
/var/gopher/output/, reads each file plus an optional sibling
<name>.bcard for metadata, and emits catalog/.gophermap files
Venusia serves. The source tree is read-only to bartleby; only
catalog/ is generated, and deleting it leaves your library
exactly as you wrote it.
By default bartleby treats an unknown extension as a text
file (item type 0). A line in bartleby.conf tells it to mark
.lhs entries as menus (item type 1) instead, so a client
navigating the catalog sees them as gopher menus and follows
them without being prompted for input:
item_types:
.lhs: "1"
bartleby >= 0.3.1 is required — earlier versions rejected
'1' as an item_types value on the theory that menus must be
directories. v0.3.1 lifted that exactly because CGI-style
script files want to declare "I am the menu" to the catalog.
Optional metadata: a sibling <name>.lhs.bcard with
`title:` / `description:` / `created:` / `updated:` shows up
under the applet's catalog entry. Without one, the catalog
entry is just the bare filename.
Pipeline, end to end:
1. you drop applets/figlet.lhs (and optionally
applets/figlet.lhs.bcard) into /var/gopher/output/
2. bartleby reads the tree and writes catalog/applets/.gophermap
with a line:
1figlet text renderer<TAB>/applets/figlet.lhs<TAB>host<TAB>port
because of the .lhs->1 mapping in bartleby.conf
3. a client navigating the catalog clicks that entry — its
type-1 prefix means "menu", so the client follows
without prompting
4. Venusia matches /applets/figlet.lhs against its
[[files.script_extension]] rule, run-cached-lhs execs
the (cached) compiled binary, the binary writes a
gophermap to stdout, and Venusia returns that verbatim
bartleby is at https://github.com/someodd/bartleby.
-------------------------------------------------------------
setting up a host
-------------------------------------------------------------
Three things have to be on the box before an applet can
serve:
* `stack`, on the serving user's PATH. The path of least
resistance is ghcup (https://www.haskell.org/ghcup/), the
official Haskell toolchain installer — it drops stack,
GHC, and cabal under ~/.ghcup/bin. You don't pick or
manage a GHC version by hand: each script pins its own
resolver on the `-- stack` line, and stack fetches the
matching GHC the first time that resolver compiles.
ghcup's job here is just to get `stack` itself onto the
system.
* the run-cached-lhs and doctest-lhs wrappers on PATH,
conventionally /usr/local/bin. Both ship in this
directory:
sudo install -m 755 run-cached-lhs doctest-lhs /usr/local/bin/
* the Venusia daemon, with a routes.toml that routes
/applets/*.lhs through run-cached-lhs. The routes.toml in
this directory is the working example; the
[[files.script_extension]] block is the part that does it.
One more, but only for applets that opt into LiquidHaskell:
the z3 CLI (`apt install z3`) and a resolver where
`liquidhaskell` resolves. Both are detailed under "optional:
liquidhaskell verification" below — applets without LH bits
need neither.
-------------------------------------------------------------
minimal template
-------------------------------------------------------------
#!/usr/bin/env stack
> -- stack script --resolver lts-22.6 --package text
hello.lhs — emit a one-line gophermap that names the
selector that resolved to this script.
> {-# LANGUAGE OverloadedStrings #-}
> module Main (main) where
>
> import qualified Data.Text as T
> import qualified Data.Text.IO as TIO
> import System.Environment (getArgs)
>
> main :: IO ()
> main = do
> args <- getArgs
> let sel = case args of (s:_) -> T.pack s
> _ -> "/applets/hello.lhs"
> TIO.putStr ("iHello from " <> sel <> "\t\t\t0\r\n")
> TIO.putStr ".\r\n"
-------------------------------------------------------------
running it locally
-------------------------------------------------------------
The `#!/usr/bin/env stack` shebang on every applet means the
file is directly executable once you set the execute bit:
chmod +x hello.lhs
./hello.lhs
stack reads the `-- stack` directive, resolves the resolver
and packages, compiles the source (or reuses its own
script-binary cache under ~/.stack/script-binaries/ if
nothing has changed), and execs the binary. Same compile,
same LH refinement checks if the file enables the plugin,
same gophermap on stdout. This is the path you work in while
writing an applet — edit, save, run, read the bytes, repeat
— with no Venusia install in the loop.
It's slower per run than the served path, though: stack
revalidates its cache on every invocation, which costs the
2-3 seconds run-cached-lhs exists to amortise away. The
wrapper caches the binary at a known location keyed by
mtime+size and execs it directly, never waking stack on a
hit, so /applets/ requests stay at ~50 ms. The shebang path
is the same applet end to end — it just pays the startup tax
each time in exchange for needing nothing but `stack` on
PATH.
That PATH caveat is the one gotcha: a non-interactive ssh
session (no `.zshrc` sourced) won't find stack by default.
Run as `PATH=/home/venusia/.ghcup/bin:$PATH ./script.lhs`, or
log in interactively first.
-------------------------------------------------------------
running doctests
-------------------------------------------------------------
Doctest is a post-compile harness: it spawns GHCi, reloads
the file, evaluates each `>>>` example, and diffs the result
against the expected output line below. That makes it a
separate step from the compile itself — distinct from the
LiquidHaskell pipeline below, which lives inside the compile
as a GHC plugin and so fires automatically every time the
file is built. The two halves are complementary: doctest
covers `>>>` examples on pure helpers, LH covers refinement
types; together they form the file's verification path.
The companion wrapper to run-cached-lhs:
doctest-lhs path/to/script.lhs
It reads the script's own "-- stack" directive (so the resolver
and --package list always match what run-cached-lhs compiles
against) and extracts the file's LANGUAGE pragmas as -X flags
for doctest's GHCi session. No per-script invocation prose to
maintain; no -XOverloadedStrings to remember.
For scripts that enable the LiquidHaskell plugin (see next
section), the wrapper rewrites the plugin pragma to an inert
comment in a temp copy before invoking doctest, so LH's
"LIQUID: SAFE ..." banner doesn't poison doctest's stdout
capture. Line numbers are preserved; doctest failures still
point at the right source line.
Exit status mirrors doctest's: 0 on a clean run, non-zero on
any failure. The wrapper lives at /usr/local/bin/doctest-lhs
alongside run-cached-lhs and uses the same directive-parsing
awk so the two stay in step.
-------------------------------------------------------------
optional: liquidhaskell verification
-------------------------------------------------------------
For applets that touch untrusted input or carry size/range
invariants, you can ask LiquidHaskell to check pure-function
refinements at compile time. Unlike doctest above, LH is a
GHC plugin: enabling it adds a check inside the compile that
run-cached-lhs (or the shebang path) runs anyway, so there
is no separate command — every cache miss re-runs the
refinements, and a failure prevents the binary from being
cached at all.
The integration is opt-in per file — the minimal template
above doesn't carry any LH bits, and applets that don't want
refinement types keep working unchanged.
Host requirements (one-time):
* `z3` CLI installed (`sudo apt install z3`). The Z3 *library*
alone isn't enough — LH shells out to the binary at build
time.
* A resolver where `liquidhaskell` resolves. As of writing,
no `lts-22/23/24` snapshot ships it; a date-pinned `nightly`
is the practical choice. The author tested
`nightly-2025-12-01` (GHC 9.12.2 + liquidhaskell 0.9.12.2)
and it works end-to-end with run-cached-lhs.
Per-file opt-in (three lines added to the script):
1. The `--package liquidhaskell` flag on the `-- stack`
directive line.
2. The plugin pragma:
{-# OPTIONS_GHC -fplugin=LiquidHaskell #-}
3. Whichever LIQUID option lines you need to silence LH's
stricter defaults. For most retrofits onto existing IO-
containing code:
{-@ LIQUID "--no-totality" @-}
{-@ LIQUID "--no-termination" @-}
Then write the refinements you want as `{-@ ... @-}` blocks
above the relevant functions. Three flavours, ordered from
least to most useful:
* Numeric/range refinements on top-level constants
(`{-@ maxSnippetLen :: {v:Int | v >= 4} @-}`) — LH
*verifies* these. Drop the constant below the bound and
compilation fails.
* `assume` annotations declare a trusted postcondition
LH won't verify but will propagate to callers. Useful
for small audited functions like a sanitiser, where the
body is obvious and the win is forcing every call site
to admit it.
* Function refinements with `{-@ name :: ... @-}` (no
`assume`) — LH *verifies* these by checking the body
against the spec. Most powerful, most fragile; the
library's measures (e.g. `len` on lists) cover lists
cleanly but not `Data.Text` in LH 0.9.12.
Failure visibility: LH errors come through the same path as
GHC errors. run-cached-lhs's `set -eu` aborts before the
binary is cached, the request's stderr surfaces the error,
and the client sees no body. Same shape, same UX, same
debugging path as a type error.
First-compile cost: LH + Z3 + the LH dep tree compile takes
~15-30 min on a small server the first time the resolver is
seen. Subsequent compiles on the same resolver are cached at
the stack-snapshot level; subsequent requests against the
same source-mtime hit the run-cached-lhs binary cache (~50 ms).
The slow path is paid once per resolver bump, not per request
and not per source edit.
LH and the two invocation paths: LH runs at compile time, not
runtime. The cached binary that ends up under
/var/cache/venusia-compiled/ is ordinary Haskell — executing
it doesn't re-check the refinements. That's not a hole; it's
how typed languages work. The check that matters happens
before the binary is written.
LH fires on both compile paths — the daemon's run-cached-lhs
invocation and the shebang path (see "running it locally"
above) — checking on first compile and skipping on unchanged
source. The two caches are independent, though: a fix you've
re-exercised via the shebang doesn't refresh the daemon's
binary under /var/cache/venusia-compiled/, which stays stale
until the next request recompiles. When in doubt about which
binary is live, edit-and-touch the source to force both.
`grep.lhs` in this directory is the worked example.
-------------------------------------------------------------
verified template
-------------------------------------------------------------
#!/usr/bin/env stack
> -- stack script --resolver nightly-2025-12-01 --package text --package liquidhaskell
verified-hello.lhs — same as the minimal template, but with
one verified refinement type and one doctest example, to
prove both halves of the verification pipeline are alive.
> {-# LANGUAGE OverloadedStrings #-}
> {-# OPTIONS_GHC -fplugin=LiquidHaskell #-}
>
> {-@ LIQUID "--no-totality" @-}
> {-@ LIQUID "--no-termination" @-}
>
> module Main (main) where
>
> import qualified Data.Text as T
> import qualified Data.Text.IO as TIO
> import System.Environment (getArgs)
>
> -- | The number of rows we emit. LH verifies the @>= 1@
> -- refinement at compile time; flip the body to 0 and
> -- compilation fails before the binary is cached. The
> -- @>>>@ example below is verified by `doctest-lhs`.
> --
> -- >>> rowCount
> -- 1
> {-@ rowCount :: {v:Int | v >= 1} @-}
> rowCount :: Int
> rowCount = 1
>
> main :: IO ()
> main = do
> args <- getArgs
> let sel = case args of (s:_) -> T.pack s
> _ -> "/applets/verified-hello.lhs"
> TIO.putStr ("iHello from " <> sel
> <> " (" <> T.pack (show rowCount) <> " row)\t\t\t0\r\n")
> TIO.putStr ".\r\n"
Two commands exercise the two halves:
./verified-hello.lhs # compile -> LH refinements checked
doctest-lhs verified-hello.lhs # >>> examples verified
-------------------------------------------------------------
limits
-------------------------------------------------------------
* Server caps in-flight connections at 256. Long-lived
streams (icecast.lhs, future radio things) count.
* Accepted sockets have Linux TCP_USER_TIMEOUT = 120s; a
silently-dead client is reaped within two minutes.
* The compile cache survives reboots. After a GHC version
upgrade (ghcup install ghc <new>), the dynamically-linked
binaries break; clear with:
sudo rm -rf /var/cache/venusia-compiled/*
* Each script's output should be self-contained (gophermap
terminator if menu, raw bytes if binary). The framework
passes stdout through verbatim with no transformation.
-------------------------------------------------------------
see also
-------------------------------------------------------------
The daemon is Venusia. The relevant features are nested
[[files.script_extension]] (0.7.0.0), $selector substitution
(0.7.0.0), trailing-slash routing equivalence (0.7.1.0),
$pathinfo path-info dispatch (0.8.0.0), $remote_ip peer-IP
substitution (0.10.0.0), and the streaming
response constructor (0.6.0.0).
https://github.com/someodd/venusia
The catalog these applets surface under is generated by
bartleby. v0.3.1 added the `item_types: .lhs: "1"` mapping
that makes .lhs catalog entries link as menus rather than as
text files — without it, clicking a .lhs row would prompt
the client to fetch raw source.
https://github.com/someodd/bartleby
To see a more complete and working set of Gopher applet
source files please see this URI:
BIN doctest-lhs 1.9KB 2026-05-19
BIN grep.lhs 18.1KB 2026-05-18
BIN icecast.lhs 2.9KB 2026-05-07
BIN run-cached-lhs 1.1KB 2026-05-19