CAN YOU WRITE SOFTWARE USING ONLINE CODE TESTS?
I was reminded of the unfortunate reality of code tests by
way of an unhappy job applicant on a Discord server. A coder
was woe-ing the hours they had spent on a code test for a
job. It got me thinking about code tests, including my own
experiences1 (well… lack thereof). Then my mind began to
speculate on just how sinister code tests could
become. Like, could you get a piece of software written
entirely from code tests? Imagine: a tech company whose
product is software that develops software out of code
tests. These code tests are embedded into job openings for
positions in companies that don’t exist. A code test for one
position at one company might involve writing a front-end
template or small API. A code test for another position at
another company might involve testing the results of the
other applicants' work. Coders apply to the jobs and
complete the tests. Regardless of the outcome they’re never
hired and get no further in the process. This cycle repeats
in endless iterations. Code gets written, but no one ever
gets paid. All the while–and unbeknownst to all who’ve
applied–a piece of software is being developed behind the
curtain.
Can it be done? Just how difficult would it be to execute
this sinister plot? (And: if I am to reason about how to
implement this scenario, is that in itself a type of code
test?) Setting up a fake company would be the easy
part. There are website generators, brand asset generators,
and website copy generators to quickly stitch together
something that looks like yet-another startup company. But
what about the code tests themselves? The scope of this
problem is beyond anything I’ve ever had to reason about. I
couldn’t tell you what language, development kit, platform,
etc., to use. It strikes me that maybe a Lisp dialect would
be effective, since I hear they are well suited for AI, and
AI would probably factor into this scenario
somehow. Nonetheless, Clojure (because it’s a Lisp dialect
and it can run on a JVM) and Haskell (for its strongly type
nature, and the fact that its used by Meta for spam
detection) would be my language candidates. (And since I
want to learn Haskell, so I guess I’d just go for that :))
But that approach is all based on the assumption that a
program can be written to intelligently write, evaluate, and
stitch together code tests. And this assumption could very
well be flawed.
So instead, I think it would be worth exploring a direction
that leverages version control. I’m become more familiar
with Github’s actions and API, so I think there could be a
solution that makes use of the platform. A piece of software
could be written (using any scripting language, really) to
write, pull, merge, lint, and build code tests. The software
could also open new repos which job applicants get invited
into for their test. I’m imagining a piece of software who’s
main job is “perception management”–to keep up the
perception that the code a job applicant is working on
hasn’t been written by a previous job applicant. The
software achieves this by moving the code around between
repositories to obfuscate the author. To kick the whole
thing off the first test could be to write a code test. And
the next code test (to be completed by another applicant for
a different “position”) could be to evaluate the test. The
cycle is endless. Broken down into discrete chunks, code
tests could be used, brick by brick, to build a piece of
software that builds software using code tests. Evil stuff!
Nobody better get any ideas…