On the hURL hack by Christopher Williams 2025-05-30 Early last year, zcrayfish discussed issues with the hURL hack in two phlog posts. The "hURL hack" is the use of a selector prefixed with `URL:`, where the rest of the selector is interpreted as a literal URL (typically used with type `h` items). Many Gopher clients understand this type of selector natively, but for clients that don’t, Gopher servers are recommended to generate an HTML page with a redirect. (See gopher://quux.org/0/Archives/Mailing%20Lists/gopher/gopher.2 002-02?/MBOX-MESSAGE/34[1] for more information.) Some problems can arise when a URL contains unsafe characters if they are not properly handled. In the first post[2] zcrayfish wrote: ``Take this working example URL that contains the quotation mark character (tabs have been replaced with pipes): - - - - - - - - - - - - - - - - - - - - - - - - - - - h|Amazing URL yay|URL:https://radar.zcrayfish.soy/"uhoh".html|gopher.zcrayfish.soy|70 - - - - - - - - - - - - - - - - - - - - - - - - - - - The problem, the quotation mark character destroys the anchor on the generated page... Now, RFC1738 says "All unsafe characters must always be encoded within a URL", and it specifically includes the quotation mark character as an example of one which is sometimes unsafe. This sounds reasonable at first—any "unsafe" character can be encoded using percent-encoding... except when it can’t. A URL such as `http://example.com?foo&bar` has an "unsafe" ampersand (it’ll break HTML if left untreated), but it can’t be percent-encoded (as `%26`) since that would change the meaning of the request. zcrayfish continues: ``Alternatively, for folks not seeking compliance with the URL RFCs, HTML entities can be used for the reserved characters which are causing issues. An HTML entity for an ampersand looks like `&` (named entity) or `&` (hexadecimal entity) or `&` (decimal entity) (strictly speaking, `&` is called a "character entity reference", while `&` and `&` are called "numeric character references", but they’re all equivalent, so use whichever you prefer). The issue with using HTML character entities in a `URL:` selector, at least on some servers like geomyidae, is that the server itself doesn’t even allow an ampersand character in a URL; geomyidae simply returns an error to the client when such a selector is requested (before zcrayfish’s post, geomyidae actually did allow ampersands, but it was broken in other ways as mentioned in the post). This means that some URLs are simply unusable on such servers. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- NOTE : I don’t mean to pick on or single out geomyidae : here. It simply happens to be one of the servers : that I’m most familiar with, partly thanks to its : easy-to-read code. It’s also the one server I : know of that suffers from this particular issue. : As zcrayfish mentions in a follow-up post, other : servers (bucktooth, gophernicus, and motsognir) are : still broken in their handling of `URL:` selectors : in various ways, even after being "fixed". =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- The other problem with this is that the selector can no longer contain an actual URL but must contain an HTML-encoded URL. A user of a server shouldn’t be responsible for HTML-encoding URLs in their gophermaps! HTML-encoding a URL in a gophermap is also fixing the problem in the wrong place. Fixing the problem in the wrong place doesn’t fix another problem: a server can still be tricked into generating "unsafe" HTML, as zcrayfish notes: ``In addition to breaking some legitimate URLs, this is a security issue which allows arbitrary code insertion, including XSS attacks. For POC point curl against any gopher server with a hURL and just add the following to the end of your URL: - - - - - - - - - - - - - - - - - - - - - - - - - - - ">alert("I%20am%20an%20alert%20box!"); - - - - - - - - - - - - - - - - - - - - - - - - - - - For the servers that generate the hURL page in HTML, anyone who tries to render it with javascript enabled is absolutely going to get a popup. So what’s the correct way to handle a `URL:` selector? The server itself must HTML-encode the URL. Full stop. A selector can then contain a standard URL with no special encoding, as in zcrayfish’s example menu item above. By requesting the selector in that example, the server should generate a link in the HTML redirect code that looks like `... ` (or alternatively using `"` or `"`). To illustrate, this site (gopher://asciz.com[3]) HTML-encodes the URL in HTML redirect pages. A request to my server with a malicious URL as in the POC above will return a safe HTML redirect page: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - No javascript popups for you! The server that this site runs on uses a shell script to handle `URL:` selectors (it uses scripts to handle a lot of stuff, in fact). The code which encodes a URL looks like the following: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - sed 's,&,\&,g; s,",\",g; s,<,\<,g; s,>,\>,g; s,'\'',\',g' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Any other Gopher server should do the same thing in whatever way is suitable for the language it’s written in (e.g., use `.replaceAll("&", "&")` in Java, or maybe some `unpack` incantation in Perl). In a follow-up post[4], zcrayfish wrote: ``BTW, now is a good time to mention (as geomyidae now does!): The hURL hack is stupid. Very stupid. Please stop using it; we’ve had a gopher-native URL scheme that is much better since 1992. It is type w. On one hand I can’t say that I disagree, but on the other hand I see some value in continuing to support `URL:` selectors in servers if only for the benefit of users of old clients that support HTML but don’t support type `w` items or `URL:` selectors (I’m looking at you, w3m!). But if you’re going to support `URL:` selectors in your Gopher server, please at least support them properly! ------------------------------------------------------------ References ------------------------------------------------------------ [1] gopher://quux.org/0/Archives/Mailing%20Lists/gopher/gopher.2002-02?/MBOX-MESSAGE/34 [2] gopher://gopher.zcrayfish.soy/1/phlog/20240209-hurls-come-back-to-bite [3] gopher://asciz.com [4] gopher://gopher.zcrayfish.soy/1/phlog/20240227-which-gopher-daemons-have-been-patched-to-fix-the-hurl-issue