K Machine Software Development Software overview: The K Machine software consists of several major pieces, the compiler, the low level system software, the Lisp language, and the various facilities which make up the programming environment. The system is to be based firmly on a solid implementation of Common Lisp, with almost the entire core of the system being written in this fairly well defined, standard language. As in other RISC type machines, the compiler plays an important role in matching the language to the hardware execution environment. With most of the system written in Common Lisp, a solid Common Lisp compiler is essential. We have nearly completed an optimising compiler, based upon the Orbit compiler from Yale, with extensive rewriting of the front end for Common Lisp and the back end for the K Machine. This compiler is based upon new compiler ideas developed within the last several years by Guy Steele, and the Scheme community with particular empasis on control structure and environment analysis. This compiler has been in use for several months, compiling the more than two dozen files which make up the K Machine cold load. The low level system software provides the functionality of an operating systems core on more conventional machines, and handles the low level storage management with respect to garbage collection that is neccessary on a Lisp Machine. The low level consists of the following modules: Trap Handler This program has to be able to save the complete state of the machine into registers, and how to write the registers into memory. It also needs to recognize a few special cases that have to be fast and can avoid having to save all the state. Individual trap handlers handle datatype, single-step, memory fault, and interrupt traps. Status: 75% finished Paging System Memory Map Virtual Memory Physical Cluster Data Table Quantum Map Memory Management There will have to be a new disk swap handler. On top of that, there will be the page fault handler itself. The page fault handler will be able to take advantage of the fact that there is no longer two levels of virtual memory mapping. This spells the demise of the infamous Page Hash Table. Since much of the information regarding clusters and region attributes will be held in hardware and not in software, an examination of the tables used for the Lambda page fault software is in order. The new page fault code will will also use a Lisp version of Naha's quantum map code with some variations. One possible variation is to keep track of the quanta's physical disk locations so that we can maximize the size of disk transfers. The quantum size will be the region granularity. Status: mostly done except for device driver or IOP interface GC The transporter will be split into two parts. One part will simply know how to copy objects from an oldspace region in an area to an appropriate copyspace region in the same area. Another part of the transporter will have to interact with the trap routine to handle real oldspace traps. They will operate in several modes depending on why we are transporting. Transporting from oldspace is different from following forwarded objects, but both go through the transporter. The scavenger, for instance, transports things differently. The GC hardware (transporter RAM) has different modes to support this feature. Status: GC RAM: done Transporter RAM: done Transporter: microcode needs to be rewrittin in Lisp, existing lisp code can probably be ported. Scavenger The scavenger can be a very high level piece of code. It simply scans memory and calls the transporter on the things that it finds. It should be able to interact directly with the page falt handler so as to improve paging performance. Status: not done, can probably be ported from old system. Existing stuff: GC.LISP #347 85 86473(8)! 03/30/87 15:52:40 robert UC-SCAVENGER.LISP #148 39 39278(8)! 02/05/87 15:50:32 RG UC-TRANSPORTER.LISP #85 41 41235(8)! 12/16/86 01:21:45 rg Storage Allocation Region Bits done Region Data done Area Data done Much of the existing Common Lisp could use rewriting. In the interest of time, this should be done as little as possible. Low level storage functions will need rewriting because they are largely written in microcode, and the hardware and storage conventions are different. This has mostly been done. Higher level functions should be collected from the old system, with minor editing to insure Common Lisp compatability and to clean up dependencies on changed architectural features (such as CDR coding). The modules that make up Common Lisp are: Simple Lists Status: low level done and in use, higher levels available from old system Arrays Status: allocation done, simple cases in use now full MAKE-ARRAY not yet implemented Sequences etc. Status: We have two implementations of this, but both need rework GENRIC.LISP #55 91 92658(8) ! 05/13/87 16:40:40 robert Generic sequences, this is written at the top of the file: ;>> In general these are abysmally slow. ;>> Many many many many special-cases need to be written. ;>> This is all very depressing. Numbers Status: mostly written, not completely tested needs transcendental functions, and random numbers, can be ported from old system old stuff: NUMDEF.LISP #13 9 8918(8) ! 09/11/85 21:28:25 Mly NUMER.LISP #82 27 26775(8) ! 10/08/86 11:23:38 robert Loader partially done, untested Streams Pathnames Hash Tables written, needs compiler work to compile efficiently Printer done Reader done Evaluator done Format Package System done Setf done The following are the minimal neccessary additions to Common Lisp in order to support program development. Error System Status: not started, should use object system, maybe port old the old code consists of: CONDITION-FLAVORS.LISP #13 16 15729(8)! 10/16/85 19:50:33 JRM EHF.LISP #293 64 64654(8)! 03/26/87 16:01:11 robert ERRMAC.LISP #26 28 27766(8) ! 02/12/87 19:07:25 RPK TRAP.LISP #33 69 70105(8) ! 01/19/87 21:22:08 RG Debugger Two levels: The set of debugger primitives will implement functions to access and modify inactive stack groups. It will need to find the current active and open frames, find the function associated with a frame, distinguish between arguments, locals, and temporaries in aframe, prepare the stack group to reinvoke or return from a frame, and insert a trap on exit frame. This is closely related to the trap handler. The user interface to the debugger primitives. We need commands to: print a backtrace of functions, print a backtrace with arguments and locals, return from a frame, reinvoke a frame, continue from CERROR. Unless someone comes up with a better idea fast, we will use the same basic interface as the Lambda. This should be mostly a Common Lisp program. Status: preliminary cross machine debuggers and steppers exist, mostly this has only been thought about. None of the existing primitives will probably be useful as the control information is completely different. The existing user interface and command loop may be applicable. Existing code: EH.LISP #395 140 143237(8)! 12/27/86 15:42:52 jrm EHBPT.LISP #9 17 16754(8) ! 12/18/85 12:22:21 khs EHC.LISP #262 83 84171(8)! 10/02/86 05:59:55 robert EHW.LISP #125 33 33210(8)! 10/07/86 15:53:58 mrc Window error handler, not generally used Processes/Scheduler/Job Control Stack groups and stack group switching on the Lambda are largely implemented in microcode and K stack groups will be quite different, this code will need rewriting. The existing scheduler may be applicable although it might benifit from some conventional operating system techniques. Processes should probably be implemented to not be flavors. Currently "Job Control" on the Lisp Machine is mostly handled through the window system. Pace felt a more conventional Job Control system was neccessary for operating the machine over terminal links. His comments: This is a UNIX-like or ITS-like terminal and process manager so that a simple serial interface can be used to control the multiple processes of the machine. The scheduler and top-level read-eval-print loop are implemented here. The documentation of the ITS job-control primitives is available on ANGEL. The SUPDUP server implements the beginnings of a job-control system. Subprojects: 1. Finish job-control support in SUPDUP server: a. Implement TYPEOUT permissions. b. Implement a command loop to handle tasks such as creating jobs, giving a job control of the tty, intercepting a keyboard command to return to the command loop, or to send special characters like ABORT. 2. Write the read-eval-print loop 3. Write a scheduler. Naha was working on this. Rubout Handler We will use the present rubout handler, but need to change it to use straight Common Lisp. Also it needs to understand scrolling terminals. Status: actually there are several old ones, nothing done yet. RH.LISP #182 62 63026(8)! 03/30/87 12:35:07 robert Make-System The present DEFSYSTEM/MAKE-SYSTEM/PATCH program is very buggy and several attempts to fix it have failed. Therefore, it is time to retire it and do something new. We need to look at the system programs for other Lisps, and pick out something. (Some people like Symbolics system 7 make-system, George says he knows of a possibility that already includes a patch system). If no one else has anything reasonable, then we can do something simple ourselves. The following modules constitute the program development environment. Zetalisp Compatibility Zetalisp compatiblity for those who need it. I expect that at some point these parts of the system will not be shipped unless a customer asks for it. (At the beginning, we may need to ship it with every system to support zwei, etc.) Flavors/Common Loops/CLOS Old flavors is out of date and should be replaced however, much of our window sytem and editor is written in it. To bring it up we would need to design the data structures and write in lisp what is now the micrcode support. We might be able to get away without doing some of the really esoteric stuff. We might even be able to do this in Common Loops. Old Flavors: FLAVOR.LISP #316 202 206152(8)! 02/17/87 15:49:10 RpK misc stuff in UC-CALL-RETURN, etc The lisp community is converging on a standard for object oriented programming oriented toward generic functions and strongly derived from New Flavors and Common Loops, we must participate in this. Graphics The current graphics primitives are an ad-hoc set based on memory mapped, raster scan, pixel displays. They are not very device-independent. They could be ported without much trouble, microcode would need rewriting. Old stuff: UC-TV.LISP #14 42 42323(8) ! 01/22/87 23:24:28 rg GRAPHICS.LISP #11 34 33811(8) ! 10/10/86 16:57:19 robert A much richer graphics set would enhance the user interface to the K machine, a particularly powerful imaging model can be found in PostScript, (furthur discussed in relation to NeWS, in window section) Window System The window system is a very large, complex system which provides much functionality while at the same time being rather intractable. I would not like to mess much with the architecture of the window system but three things should be done with it: 1. Code should be converted to Common Lisp as much as possible. (John Mann started doing this at one point) 2. It should be rebased on the Common Lisp object system. At the least, generic functional interfaces should be provided, to help customers migrate to that style. If old style flavor syntax compatibility is maintained this change could be mostly transparent to users. 3. The lower level screen management and graphics should be replaced with a network window system (preferebly NeWS), which would allow using the machine from other machines (other Lisp machines, Suns, PCs etc). (See NeWS probaganda.) This change could be mostly transparent to users. Old window system: TVDEFS.LISP #297 45 45306(8)! 11/12/86 16:02:40 gjc SHEET.LISP #583 109 110858(8)! 03/30/87 12:23:59 robert SHWARM.LISP #370 87 88794(8)! 12/31/86 14:09:49 EFH BASWIN.LISP #577 81 82659(8)! 03/20/86 16:01:42 JRM BASSTR.LISP #411 80 81658(8)! 05/27/87 19:44:47 pld UC-TV.LISP #14 42 42323(8) ! 01/22/87 23:24:28 rg GRAPHICS.LISP #11 34 33811(8) ! 10/10/86 16:57:19 robert SCRMAN.LISP #169 35 34892(8)! 02/05/86 18:15:00 pace UC-TRACK-MOUSE.LISP #20 20 20229(8)!@ 05/30/86 13:55:05 dg MOUSE.LISP #267 57 57625(8)! 10/10/86 10:06:12 jrm STREAM.LISP #152 26 25994(8)! 05/06/86 20:25:01 RMS MENU.LISP #115 60 60831(8)! 03/30/87 13:16:35 robert FRAME.LISP #172 43 43532(8)! 04/22/86 23:35:52 dg CHOICE.LISP #130 82 83896(8)! 07/08/86 14:10:34 mrc SCRED.LISP #117 68 69063(8)! 04/22/86 23:35:02 dg SCROLL.LISP #183 46 46943(8)! 04/22/86 23:35:34 dg SYSMEN.LISP #196 43 43412(8)! 10/02/86 04:01:50 robert WHOLIN.LISP #105 29 29468(8)! 03/11/87 21:16:54 jrm TSCROL.LISP #76 27 27380(8) ! 05/31/86 04:36:10 gjc TYPWIN.LISP #123 29 29166(8)! 10/12/85 22:33:17 rg Editor Zwei will depend on both flavors and the old window system. Inspector This is not hard, it depends on the window system. File System Need a new one. Tape System